is there any performance gain in using aiobotocore compared to botocore?

37 Views Asked by At

I have dataset in the huggingface format, and i can save that to some s3 path ( sample code below) . aiobotocore is async client, but is there any perfomance gain in using this instead of botocore. when running following code in aws sagemaker , is there any advantage of using this package?

impot aiobotocore
from datasets import Dataset

#process data 
 .....

#upload to s3
hugginface_dataset.save_to_disk('s3_path', storage_options={'session: aiobotocore.AioSession()})

I'm currently using botocore but planning to use aiobotocore client.

0

There are 0 best solutions below