In tensorflow, Do I have to set something special to ignore zero padding value when training? Or is it automatic?

Question

I want to train sequence data to Rnn base model with some zero paddings using tensorflow.

And I want model to ignore 0 values when training.

Do I have to set parameters to do that? or Does model automatically ignore zeros?

Thanks,,

Answer 1

It is not automatic, You should introduce Masking to achieve this. It means how layers are able to know when to ignore certain timesteps in sequence inputs.

You can introduce it in three ways

• You can add a tf.keras.layers.Masking layer
• You can configure a tf.keras.layers.Embedding layer with mask_zero=True
• You can pass a mask argument manually when calling layers that support this argument

For more information you can refer Masking and padding with Keras guide.