how to upload multiple files from local to one file in s3

525 Views Asked by At

I have hundreds of files in my local directory and I want to treat them as parts of a single file and upload them to s3, I want them as a single file in s3. ideally, I want to use s3fs or boto3 to accomplish this but any other approach is also good.

1

There are 1 best solutions below

0
On BEST ANSWER

There is no provided command that does that, so your options are:

  • Combine the files together on your computer (eg using cat) and then upload a single file using boto3, or
  • In your Python code, successively read the contents of each file and load it into a large string, then provide that string as the Body for the boto3 upload (but that might cause problems if the combined size is huge)