`Tensor is not and element of this graph` error when using Keras model in multithread in python

When a python script containing a keras model, it will work fine when the script is executed without any multithreading/multiprocessing. However, once multithreading is used and a keras model needs to execute in a new thread, it will cause problems. Here is an error message that I encountered.

  File "/home/ubuntu/venv/lib/python3.6/site-packages/keras/engine/training.py", line 1797, in predict
    self._make_predict_function()
  File "/home/ubuntu/venv/lib/python3.6/site-packages/keras/engine/training.py", line 1009, in _make_predict_function
    **kwargs)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 2499, in function
    return Function(inputs, outputs, updates=updates, **kwargs)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 2442, in __init__
    with tf.control_dependencies(self.outputs):
  File "/home/ubuntu/venv/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 4304, in control_dependencies
    return get_default_graph().control_dependencies(control_inputs)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 4017, in control_dependencies
    c = self.as_graph_element(c)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3035, in as_graph_element
    return self._as_graph_element_locked(obj, allow_tensor, allow_operation)
  File "/home/ubuntu/venv/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3114, in _as_graph_element_locked
    raise ValueError("Tensor %s is not an element of this graph." % obj)
ValueError: Tensor Tensor("predictions/concat:0", shape=(?, 100, 16), dtype=float32) is not an element of this graph.

Solution

after loading the weights, call `_make_predict_function` method for the model returned. Here is an example

self.model.load_weights(weight_path)
self.model._make_predict_function()

after doing this, the keras model will be properly initialized even in a new thread environment and the error will disappear.