How to do k-Fold Cross Validation with tf.data.Dataset API?

Question

It easy to do k-Fold Cross Validation by using scikit-learn package. There we separately use data and labels. However, here combine data and labels before feeding into model. like:

 tf.data.Dataset.from_tensor_slices((X, Y))
        .shuffle(batch_size * 100)
        .batch(batch_size)

In this case how to do K-fold cross validation?