Im trying to load my Publaynet dataset from s3 bucket to data bricks using huggingface datasets like this:
dataset_id = "/dbfs/mnt/ocr/dataset/publaynet"
dataset = load_dataset(dataset_id, data_files={"train": "/dbfs/mnt/ocr/dataset/publaynet/train.json", "validation": "/dbfs/mnt/ocr/dataset/publaynet/val.json"}, split="train", cache_dir="./cache")
My S3 bucket is in formal like below screenshot:
Im getting this error in databricks:
