Loading Multi-Label Image Dataset with Tensorflow

Question

So, I recently started learning AI and Tensorflow, and I've been working on several projects from the Tensorflow Tutorials. All of these projects have already been labeled, and I didn't find a multi-label tutorial for image classification that suited my needs for my current project.

I acquired a dataset on Kaggle that wasn't labeled, so I had to create the labels myself. Following the structure from previous tutorials, I organized the directory like this:

Class A
  * Image-1
  * Image-2
Class B
  * Image-1
  * Image-3
Class C
  * Image-4

Each image in my dataset can belong to either one or two classes out of a dozen available classes.

After that, I started loading my dataset using tf.keras.utils.image_dataset_from_directory with the inferred option and created a Sigmoid endpoint along with BinaryCrossentropy loss. However, I realized a bit late that my data didn't belong to two classes but were separated into two images with the same data but a different class. This means that when I load it with categorical labels, I get [1,0,0,0,0] for Image-1 of Class A and [0,1,0,0,0] for the same Image in the Class B directory. I initially thought that inferred would take multi-label into consideration, but it seems not to.

All the references for multi-label classification with TensorFlow seem a bit old and were before the appearance of image_dataset_from_directory. Therefore, I would like to know if there is an easier way to load my dataset than creating and using a .csv file.

Thanks in advance for the help.