Upload AsyncRead to s3 rust

Question

I am try to upload AsyncRead trait to s3, but I can not figure out how to convert the AsyncRead object to ByteStream which the library (aws-sdk-s3) know how to work with.
The signature of my function is like this:

async fn upload(&self, key: String, data: &mut impl AsyncRead) -> Result<(),Error>;

let byte_stream = codec::FramedRead::new(read, codec::BytesCodec::new()).map(|r| r.freeze());
let res = s3
    .put_object()
    .key(id)
    .body(ByteStream::new(byte_stream))
    .send()
    .await;

The errors that I got are:

• the trait bound &impl 'async_trait + AsyncRead: AsyncRead is not satisfied
• FramedRead<&impl 'async_trait + AsyncRead, BytesCodec> is not an iterator

I found this and tried to implement it like the "good way" section but I got compile errors.

I also looked at the documentation and found nothing that can help with the conversion (found only the other way around).

How can I upload an AsyncRead object to s3?

Answer 1

It's been a long wait for this feature. Unfortunately as of the time of writing the only way is to use a multi-part upload strategy.

Take a look at this issue: https://github.com/awslabs/aws-sdk-rust/discussions/361

And here's an example of multi-part upload where you can read your AsyncRead in chunks and transfer them one by one: https://github.com/awslabs/aws-sdk-rust/blob/main/examples/examples/s3/src/bin/s3-multipart-upload.rs