How to split a tensorflow dataset into train, test and validation in a Python script?

Question

On a jupyter notebook with Tensorflow-2.0.0, a train-validation-test split of 80-10-10 was performed in this way:

import tensorflow_datasets as tfds
from os import getcwd
splits = tfds.Split.ALL.subsplit(weighted=(80, 10, 10))

filePath = f"{getcwd()}/../tmp2/"
splits, info = tfds.load('fashion_mnist', with_info=True, as_supervised=True, split=splits, data_dir=filePath)

However, when trying to run the same code locally I get the error

AttributeError: type object 'Split' has no attribute 'ALL'

I have seen I can create two sets in this way:

splits, info = tfds.load('fashion_mnist', with_info=True, as_supervised=True, split=['train[:80]','test[80:90]'], data_dir=filePath)

but I do not know how I can add a third set.

Answer 1

tfds.Split.ALL.subsplit or tfds.Split.TRAIN.subsplit apparently are deprecated and no longer supported.

Some of the datasets are already split between train and test. In this case I found the following solution (using for example the fashion MNIST dataset):

splits, info = tfds.load('fashion_mnist', with_info=True, as_supervised=True,
split=['train+test[:80]','train+test[80:90]', 'train+test[90:]'],
data_dir=filePath)
(train_examples, validation_examples, test_examples) = splits

EDIT AFTER COMMENTS

The previous code had some errors. First of all, this official link says:

Full dataset ('all'): 'all' is a special split name corresponding to the union of all splits (equivalent to 'train+test+...')

but when I tried it did not work. all would be helpful but there is an alternative. The error in the previous code is that the % must be used and that it must be specified for each set. I modified the code in this way:

import tensorflow_datasets as tfds
splits, info = tfds.load('fashion_mnist', with_info=True, as_supervised=True,
split=['train[:80%]+test[:80%]','train[80%:90%]+test[80%:90%]', 'train[90%:]+test[90%:]'],
data_dir='./')
#(train_examples, validation_examples, test_examples) = splits

for el in splits:
    print(el.cardinality())

which prints:

tf.Tensor(56000, shape=(), dtype=int64)
tf.Tensor(7000, shape=(), dtype=int64)
tf.Tensor(7000, shape=(), dtype=int64)

Answer 2

In the case of rock_paper_scissor dataset on tfds it works for me:

splits = ['train+test[:80]', 'train+test[80:90]', 'train+test[90:]']

splits, info = tfds.load( 'rock_paper_scissors', split=splits, as_supervised=True, with_info=True)

(train_examples, validation_examples, test_examples) = splits

num_examples = info.splits['train'].num_examples
num_classes = info.features['label'].num_classes