How to use tf.Dataset in Keras model.fit without specifying targets?

Question

I want to use an AutoEncoder model with Keras functional API. Also I want to use tf.data.Dataset as an input pipeline for the model. However, there is limitation that I can pass the dataset to the keras.model.fit only with tuple (inputs, targets) accroding to the docs:

Input data. It could be: A tf.data dataset. Should return a tuple of either (inputs, targets) or (inputs, targets, sample_weights).

So here is the question: can I pass the tf.data.Dataset without repeating inputs like that (inputs, inputs) and more like (inputs, None). And if I can't, will the repeated inputs double the GPU memory for my model?

Answer 1

You can use map() to return your input twice:

import os
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
import tensorflow as tf
from tensorflow.keras import Sequential
from tensorflow.keras.layers import Conv2D, MaxPool2D, Conv2DTranspose, Reshape
from functools import partial

(xtrain, _), (xtest, _) = tf.keras.datasets.mnist.load_data()

ds = tf.data.Dataset.from_tensor_slices(
    tf.expand_dims(tf.concat([xtrain, xtest], axis=0), axis=-1))

ds = ds.take(int(1e4)).batch(4).map(lambda x: (x/255, x/255))

custom_convolution = partial(Conv2D, kernel_size=(3, 3),
                             strides=(1, 1),
                             activation='relu',
                             padding='same')
custom_pooling = partial(MaxPool2D, pool_size=(2, 2))

conv_encoder = Sequential([
    custom_convolution(filters=16, input_shape=(28, 28, 1)),
    custom_pooling(),
    custom_convolution(filters=32),
    custom_pooling(),
    custom_convolution(filters=64),
    custom_pooling()
    ])

# conv_encoder(next(iter(ds))[0].numpy().astype(float)).shape
custom_transpose = partial(Conv2DTranspose,
                           padding='same',
                           kernel_size=(3, 3),
                           activation='relu',
                           strides=(2, 2))

conv_decoder = Sequential([
    custom_transpose(filters=32, input_shape=(3, 3, 64), padding='valid'),
    custom_transpose(filters=16),
    custom_transpose(filters=1, activation='sigmoid'),
    Reshape(target_shape=[28, 28, 1])
    ])

conv_autoencoder = Sequential([
    conv_encoder,
    conv_decoder
    ])

conv_autoencoder.compile(loss='binary_crossentropy', optimizer='adam')

history = conv_autoencoder.fit(ds)

2436/2500 [============================>.] - ETA: 0s - loss: 0.1282
2446/2500 [============================>.] - ETA: 0s - loss: 0.1280
2456/2500 [============================>.] - ETA: 0s - loss: 0.1279
2466/2500 [============================>.] - ETA: 0s - loss: 0.1278
2476/2500 [============================>.] - ETA: 0s - loss: 0.1277
2487/2500 [============================>.] - ETA: 0s - loss: 0.1275
2497/2500 [============================>.] - ETA: 0s - loss: 0.1274
2500/2500 [==============================] - 14s 6ms/step - loss: 0.1273

Answer 2

"the repeated inputs double the GPU memory for my model?"; Generally , the dataset pipeline run on CPU, not on GPU.

For your AutoEncoder model, if you want to use a dataset that only contains the examples without labels, you can use custom training :

def loss(model, x):

    y_ = model(x, training=True)           # use x as input

   return loss_object(y_true=x, y_pred=y_) # use x as label (y_true)

with tf.GradientTape() as tape:
   loss_value = loss(model, inputs)

If it is necessary to use the fit() method, you can subclass keras.Model and overrides the train_step() method link . (I did not verify this code ) :

class CustomModel(keras.Model):
def train_step(self, data):

    x = data
    y = data  # the same data as label ++++

    with tf.GradientTape() as tape:
        y_pred = self(x, training=True)  # Forward pass
      
        loss = self.compiled_loss(y, y_pred, regularization_losses=self.losses)

    # Compute gradients
    trainable_vars = self.trainable_variables
    gradients = tape.gradient(loss, trainable_vars)
    # Update weights
    self.optimizer.apply_gradients(zip(gradients, trainable_vars))
    # Update metrics (includes the metric that tracks the loss)
    self.compiled_metrics.update_state(y, y_pred)
    # Return a dict mapping metric names to current value
    return {m.name: m.result() for m in self.metrics}

Answer 3

In TensorFlow 2.4, I've got a dataset that returns a tuple of one element, ie (inputs,), which is training just fine. The only caveat is of course that you cannot pass a loss or metrics to model.compile, but must instead use the add_loss and add_metric APIs somewhere in the model.