How to use S3FS copy with recursive on folders?

124 Views Asked by At

does s3fs.copy(src, target, recursive=True) work for folders? https://s3fs.readthedocs.io/en/latest/api.html. delete(srcfolder, recursive=True) worked for me so I thought copy would too.

I know my files exist; the aws s3 cp command works. This copied the files in srcfolder to target folder.

aws s3 cp srcfolder  targetfolder --recursive

I tried

s3_fs.copy(key, target_key, recursive=True, on_error='raise')

but this always tells me The specified key does not exist. Checking s3_fs.exists(key): returns true. Based on all the other questions being answered on stack overflow it makes me think that there's no actual recursive file copying built in to python in either boto3 or s3fs.

If I do s3_fs.ls first and then copy the files 1 by 1 it works.

0

There are 0 best solutions below