Keras LSTM input shape is wrong

Question

I am trying to create simple RNN in Keras which will learn over this dataset:

x_train = [
    [0,0,0,1,-1,-1,1,0,1,0,...,0,1,-1],
    [-1,0,0,-1,-1,0,1,1,1,...,-1,-1,0],
    ...
    [1,0,0,1,1,0,-1,-1,-1,...,-1,-1,0]
]

which 1 means an increase in one metric and -1 means decrease in it and 0 means no change in the metric. Each array has 83 items for 83 metrics and the output (labels) for each array is a categorical array that shows the effect of these metrics on a single metric:

[[ 0.  0.  1.]
 [ 1.  0.  0.],
 [ 0.  0.  1.],
 ...
 [ 0.  0.  1.],
 [ 1.  0.  0.]]

I used Keras and LSTM in the following code:

def train(x, y, x_test, y_test):
    x_train = np.array(x)
    y_train = np.array(y)
    print x_train.shape
    y_train = to_categorical(y_train, 3)
    model = Sequential()
    model.add(LSTM(128,input_dim=83, input_length=3))
    model.add(Dropout(0.5))
    model.add(Dense(3, activation='softmax'))
    opt = optimizers.SGD(lr=0.1, decay=1e-2)
    model.compile(loss='categorical_crossentropy',
            optimizer=opt,
            metrics=['accuracy'])
    model.fit(x_train, y_train, batch_size=128, nb_epoch=200)

The output of line print x_train.shape is (1618, 83) and when i run my code I get this error:

Traceback (most recent call last):
  File "temp.py", line 171, in <module>
    load()
  File "temp.py", line 166, in load
    train(x, y, x_test, y_test)
  File "temp.py", line 63, in train
    model.fit(x_train, y_train, batch_size=128, nb_epoch=200)
  File "/usr/local/lib/python2.7/dist-packages/keras/models.py", line 652, in fit
    sample_weight=sample_weight)
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 1038, in fit
    batch_size=batch_size)
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 963, in _standardize_user_data
    exception_prefix='model input')
  File "/usr/local/lib/python2.7/dist-packages/keras/engine/training.py", line 100, in standardize_input_data
    str(array.shape))
Exception: Error when checking model input: expected lstm_input_1 to have 3 dimensions, but got array with shape (1618, 83)

I don't want to use Embedding and want to add input_shape to the LSTM layer.

Answer 1

LSTM is a recurrent layer, meaning the input data has to be three dimensional, which corresponds to a two-dimensional input shape. In practice this means that the data must have shape (num_samples, timesteps, features) and the input shape must be (timesteps, features).

In your case you are missing the timesteps dimension in both your data and input shape.