Model output "Tensor("activation_9/activation_9/Identity:0", shape=(?, 6), dtype=float32)" has invalid shape

Question

I am getting this error when I am trying to build an DQN model but I am getting this error:

ValueError                                Traceback (most recent call last)
<ipython-input-41-42c80ec471c2> in <module>()
      1 # TODO - Select the parameters for the Agent and the Optimizer
      2 dqn = DQNAgent(model=model, nb_actions=nb_actions,
----> 3                memory=memory)
      4 dqn.compile(Adam(lr=.00025), metrics=['mae'])

/usr/local/lib/python3.7/dist-packages/rl/agents/dqn.py in __init__(self, model, policy, test_policy, enable_double_dqn, enable_dueling_network, dueling_type, *args, **kwargs)

ValueError: Model output "Tensor("activation_14/activation_14/Identity:0", shape=(?, 6), dtype=float32)" has invalid shape. DQN expects a model that has one dimension for each action, in this case 6.

I´ve had some problems with the version in keras-rl and tensorflow, so these are the version I am using:

tensorflow==1.13.1

Keras==2.2.4

keras-rl2==1.0.4

The code looks like this:

from __future__ import division

from PIL import Image
import numpy as np
import gym

from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Activation, Flatten, Convolution2D, Permute
from tensorflow.keras.optimizers import Adam
import tensorflow.keras.backend as K

from rl.agents.dqn import DQNAgent
from rl.policy import LinearAnnealedPolicy, BoltzmannQPolicy, EpsGreedyQPolicy
from rl.memory import SequentialMemory
from rl.core import Processor
from rl.callbacks import FileLogger, ModelIntervalCheckpoint

INPUT_SHAPE = (84, 84)
WINDOW_LENGTH = 4

env_name = 'SpaceInvaders-v0'
env = gym.make(env_name)
nb_actions = env.action_space.n

input_shape = (WINDOW_LENGTH,) + INPUT_SHAPE
model = Sequential()
model.add(Permute((2, 3, 1), input_shape=input_shape))

model.add(Convolution2D(32, (8, 8), strides=(4, 4)))
model.add(Activation('relu'))
model.add(Convolution2D(64, (4, 4), strides=(2, 2)))
model.add(Activation('relu'))
model.add(Convolution2D(64, (3, 3), strides=(1, 1)))
model.add(Activation('relu'))

model.add(Flatten())
model.add(Dense(512))
model.add(Activation('relu'))
model.add(Dense(nb_actions))
model.add(Activation('linear'))
print(model.summary())

memory = SequentialMemory(limit=1000000, window_length=WINDOW_LENGTH)

dqn = DQNAgent(model=model, nb_actions=nb_actions, memory=memory)

I am using tensorflow==1.13 because I have the following problem with some _keras_shape in the DQNAgent too:

'Tensor' object has no attribute '_keras_shape'

Could someone tell me what I am doing wrong?