I cannot get the input shape right

Question

The following code gives me an input error and i cannot figure it out.

import tensorflow as tf
import neural_structured_learning as nsl
.
.
.
b_size = 132
m = tf.keras.Sequential()
m.add(tf.keras.layers.Dense(980, activation = 'relu', input_shape = (2206,2,)))

m.add(tf.keras.layers.Dense(560, activation = 'relu'))
m.add(tf.keras.layers.Dense(10, activation = 'softmax'))

adv_config = nsl.configs.make_adv_reg_config(multiplier=0.2, adv_step_size=0.5)
adv_model = nsl.keras.AdversarialRegularization(m, adv_config=adv_config)

adv_model.compile(optimizer = "adam",
                 loss = "sparse_categorical_crossentropy",
                 metrics = ['accuracy'])
adv_model.fit({"feature" : x_Train, "label" : y}, epochs = 50, batch_size=b_size)

My x_Train has shape (5002, 2206, 2) (5002 samples of size (2206,2)). I have tried to add a Flatten() layer at the beginning but it gives me a object of type 'NoneType' has no len() error, even though this works perfectly with tf.keras. I also have tried different shapes for the input but none of them work. So it throws me one of the following errors

KeyError: 'dense_115_input'

ValueError: Input 0 of layer sequential_40 is incompatible with the layer: expected axis -1 of input shape to have value 2206 but received input with shape [None, 2206, 2]

TypeError: object of type 'NoneType' has no len()

Answer 1

To train an NSL model with an input dictionary (like your {"feature" : x_Train, "label" : y}), the base model has to know which feature(s) in the dictionary to look at.

One way to specify the feature names is to add an Input layer:

m = tf.keras.Sequential()
m.add(tf.keras.Input(name="feature", shape=(2206, 2)))

Also as this answer pointed out, the input feature has to be flatten before passing to dense layers:

m.add(tf.keras.layers.Flatten())
m.add(tf.keras.layers.Dense(...))

Answer 2

If you want to use a dense layer the input should be (5002, 2206*2), i.e a matrix. Maybe the simplest solution is to reshape your input x_train before the "fit".

Alternatively, you can use a TimeDistributed layer (see here), but the usage of this kind of layer depends on the physical meaning behind the input dimensions. Basically, TimeDistributed applies a certain operation many times, in your case twice.

Hoping this can help you.