How can I apply imgaug augmentation to a tensorflow dataset created with dataset_from_directory?

Question

I'm curently trying to apply some data augmentation using imgaug to my existing trainDataset.

The dataset is created using dataset_from_directory as shown below.

trainDataset = tf.keras.utils.image_dataset_from_directory(
    directory,
    labels='inferred',
    label_mode='int',
    class_names=classNames,
    color_mode='rgb',
    batch_size=64,
    image_size=(224, 224),
    shuffle=True,
    seed=seed,
    validation_split=0.15,
    subset='training',
    interpolation='bilinear',
    follow_links=False,
    crop_to_aspect_ratio=False
)

The imgaug I'm trying to apply to the dataset is shown below

augmenter = iaa.Sequential([
    iaa.Fliplr(0.5),
    iaa.Affine(rotate=(-10, 10)),
    iaa.Affine(scale={"x": (0.8, 1.2), "y": (0.8, 1.2)}),
    iaa.Crop(percent=(0, 0.1)),
    iaa.Sometimes(0.5, iaa.GaussianBlur(sigma=(0, 0.5))),
    iaa.AdditiveGaussianNoise(loc=0, scale=(0.0, 0.05*255), per_channel=0.5),
    iaa.Multiply((0.8, 1.2), per_channel=0.2),
    iaa.AddToHueAndSaturation((-20, 20))
])

I cannot for the life of me figure out how to actually apply this to my dataset. I've tried using map but it doesn't work as the augmenter is expecting a numpy array? Any help would be appreciated :)

p.s this is my first time posting so apologies if I've left out anything important

Update 1:

I've now changed my code a bit so the dataset isn't being split before applying any augmentation and changed around some stuff that was recommended to me.

dataset = tf.keras.utils.image_dataset_from_directory(
    directory,
    labels='inferred',
    label_mode='int',
    class_names=classNames,
    color_mode='rgb',
    batch_size=64,
    image_size=(224, 224),
    shuffle=True,
    seed=seed,
    interpolation='bilinear',
    follow_links=False,
    crop_to_aspect_ratio=False
)

augmenter = iaa.Sequential([
    iaa.Fliplr(0.5),
    iaa.Affine(rotate=(-10, 10)),
    iaa.Affine(scale={"x": (0.8, 1.2), "y": (0.8, 1.2)}),
    iaa.Crop(percent=(0, 0.1)),
    iaa.Sometimes(0.5, iaa.GaussianBlur(sigma=(0, 0.5))),
    iaa.AdditiveGaussianNoise(loc=0, scale=(0.0, 0.05*255), per_channel=0.5),
    iaa.Multiply((0.8, 1.2), per_channel=0.2),
    iaa.AddToHueAndSaturation((-20, 20))
])

def augment(image, label):
    image = tfds.as_numpy(image)
    image = augmenter.augment_image(image)
    image = tf.convert_to_tensor(image, dtype=tf.float32)
    return image, label

augDataset = dataset.map(augment)

I'm now getting the following error and have no clue what it means.

You must feed a value for placeholder tensor 'args_0' with dtype float and shape [?,224,224,3][[{{node args_0}}]]

Any help is appreciated, thanks :)

Answer 1

It seems you are using older TensorFlow version code along with few deprecated augmentation API's.

You can use Image preprocessing or Image data augmentation layers for the augmentation. These layers can be defined inside the model after fetching the dataset using image_dataset_from_directory as below:

classifier = tf.keras.Sequential([

##data augmention layers    
  layers.Resizing(IMG_SIZE, IMG_SIZE),
  layers.Rescaling(1./255),
  layers.RandomFlip("horizontal"),
  layers.RandomZoom(0.1),
  
#model building layers
  layers.Conv2D(32, (3, 3), input_shape = (IMG_SIZE, IMG_SIZE, 3), activation = 'relu'),
  layers.MaxPooling2D(pool_size = (2, 2)),
  layers.Conv2D(32, (3, 3), activation = 'relu'),
  layers.MaxPooling2D(pool_size = (2, 2)),
  layers.Flatten(),
  layers.Dense(units = 128, activation = 'relu'),
  layers.Dense(units = 1, activation = 'sigmoid')
])

Please refer this gist for your reference.