KerasRL : Value Error: Tensor must be from same graph as Tensor

Question

I am trying to build a RL model to play the Atari Pinball game while following Nicholas Renotte's video. However, when I try to build the final KerasRL model I get the following error :

ValueError: Tensor("dense/kernel/Read/ReadVariableOp:0", shape=(256, 9), dtype=float32) must be from the same graph as Tensor("dense_4/Relu:0", shape=(None, 256), dtype=float32) (graphs are <tensorflow.python.framework.ops.Graph object at 0x000001DA9F3E0A90> and FuncGraph(name=keras_graph, id=2038356824176)).

The code:

def build_model(height, width, channels, actions):
    model = Sequential()
    model.add(Convolution2D(32, (8,8), strides=(4,4), activation='relu', input_shape=(3,height, width, channels)))
    model.add(Convolution2D(64, (4,4), strides=(2,2), activation='relu'))
    model.add(Convolution2D(64, (3,3), activation='relu'))
    model.add(Flatten())
    model.add(Dense(512, activation='relu'))
    model.add(Dense(256, activation='relu'))
    model.add(Dense(actions, activation='linear'))
    return model

height, width, channels = env.observation_space.shape
actions = env.action_space.n
model = build_model(height, width, channels, actions)

from rl.agents import DQNAgent
from rl.memory import SequentialMemory
from rl.policy import LinearAnnealedPolicy, EpsGreedyQPolicy

def build_agent(model, actions):
    policy = LinearAnnealedPolicy(EpsGreedyQPolicy(), attr='eps', value_max=1., value_min=.1, value_test=.2, nb_steps=10000)
    memory = SequentialMemory(limit=1000, window_length=3)
    dqn = DQNAgent(model=model, memory=memory, policy=policy,
                  enable_dueling_network=True, dueling_type='avg', 
                   nb_actions=actions, nb_steps_warmup=1000
                  )
    return dqn

dqn = build_agent(model, actions)
dqn.compile(Adam(lr=1e-4))

The error pops up when I call the build_agent function.

I tried using tf.keras.backend.clear_session() but that didn't help.

Answer 1

You can perhaps try adding this before your build_model:

import tensorflow as tf
tf.compat.v1.disable_eager_execution()

def build_model(height, width, channels, actions):

I was also trying to follow Nicholas Renotte's video. For more in depth answer, check this link. There will be a warning, but at least it runs.

Answer 2

He literally adresses this exact error 3mn later in that same video if you care waching it.

Basically deleting the model (del model) and rebuildint it does the trick