I have dataset in the huggingface format, and i can save that to some s3 path ( sample code below) . aiobotocore is async client, but is there any perfomance gain in using this instead of botocore. when running following code in aws sagemaker , is there any advantage of using this package?
impot aiobotocore
from datasets import Dataset
#process data
.....
#upload to s3
hugginface_dataset.save_to_disk('s3_path', storage_options={'session: aiobotocore.AioSession()})
I'm currently using botocore but planning to use aiobotocore client.