There are not much example of using
tf.keras.layers.GaussianDropout
in TensorFlow 2, and I am just converting my code from Tensorflow1.15 to Tensorflow 2, and having some difficulty to understand the new way of coding in TF2.
So, can anyone please guide me thru by giving an example of how to use
tf.keras.layers.GaussianDropout
in tensorflow2, I planning to implement it into a deep GRU network.
I have tried to append it by putting it as a hidden layer, as shown below, but I got an error saying it expects an input tensor, but my case is a GRU cell:
TypeError: Failed to convert elements of <keras.layers.rnn.gru_v1.GRUCell object at 0x7fad6c09f070> to Tensor. Consider casting elements to a supported type. See https://www.tensorflow.org/api_docs/python/tf/dtypes for supported TF dtypes.
# TF2 approach to define deep GRU
# Define the GRU Cell
for i in range(num_gru_layer):
gru_cell = tf.keras.layers.GRUCell(units=num_neurons)
gru_cells.append(gru_cell)
gru_cells = [tf.keras.layers.GaussianDropout(dropout_rate)(cell) for cell in gru_cells]
I thinking if this possible to do something like normal dropout with a dropout wrapper, as shown below where I can just append it as a hidden layer, but I searched there is no such wrapper for gaussian dropout layer. Appreciate for any guidance.
for i in range(num_gru_layer):
gru_cell = tf.keras.layers.GRUCell(units=num_neurons)
dropout_gru = tf.compat.v1.nn.rnn_cell.DropoutWrapper(gru_cell, input_keep_prob = 1.0-dropout_rate)
gru_cells.append(dropout_gru)
Ok, I found a way to use
tf.keras.layers.GaussianDropout
, so jotting this down for future reference, for the case where the TensorFlow code is newly converted from TF1.x to TF2.x.Based on tf.keras.layers.GaussianDropout documentation,
there are 2 arguments for
tf.keras.layers.GaussianDropout
, namely "rate" and "seed", which user can assigned accordingly.While under "Call arguments", it specifies the way we use this API, which mentioning "inputs" and "training". As inspired by the way how we connect one layer to another in one of the examples, thus
tf.keras.layers.GaussianDropout
can be implemented as below:where 'input' is the tensor to be fed into the deep GRU network, it will be fed into the Gaussian dropout layer, while the output of
tf.keras.layers.GaussianDropout
will have the same shape as the input (based on documentation).For clarity, below is how it will look like in real deep GRU network.