FileInputStream vs InputStream for ResetExceptions in Amazon S3 file uploading

62 Views Asked by At

I'm facing a problem in my code when it tries to upload to AWS S3. Currently, many of the attempts fail due to the following error:

Caused by: com.amazonaws.ResetException: The request to the service failed with a retryable reason, but resetting the request input stream has failed. See exception.getExtraInfo or debug-level logging for the original failure that caused this retry.; If the request involves an input stream, the maximum stream buffer size can be configured via request.getRequestClientOptions().setReadLimit(int)

Looking at the AWS documentation for avoiding ResetExceptions, it notes that best practice is to "provide data by using a File or FileInputStream." In the code I'm using an InputStream, which FileInputStream extends. Would changing the type from InputStream to FileInputStream make any difference?

My alternative option would be to setReadLimit, but was wondering if the above change would be sufficient. Thank you!

1

There are 1 best solutions below

1
smac2020 On

I see you are using the AWS SDK for Java V1. This is not best practice and this SDK version is on the path to deprecation.

Announcing end-of-support for AWS SDK for Java v1.x effective December 31, 2025

This is stated in the doc topic you referenced. See note at top.

enter image description here

The Best practice to upload files using AWS SDK for Java V2 is to use Transfer Manager.

See docs here in the V2 Developer Guide:

Transfer files and directories with the Amazon S3 Transfer Manager

Here is the Code Example:

package com.example.s3.transfermanager;

// snippet-start:[s3.tm.java2.uploadfile.import]
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import software.amazon.awssdk.transfer.s3.S3TransferManager;
import software.amazon.awssdk.transfer.s3.model.CompletedFileUpload;
import software.amazon.awssdk.transfer.s3.model.FileUpload;
import software.amazon.awssdk.transfer.s3.model.UploadFileRequest;
import software.amazon.awssdk.transfer.s3.progress.LoggingTransferListener;

import java.net.URL;
import java.nio.file.Paths;
import java.util.UUID;
// snippet-end:[s3.tm.java2.uploadfile.import]

/**
 * Before running this Java V2 code example, set up your development
 * environment, including your credentials.
 *
 * For more information, see the following documentation topic:
 *
 * https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/get-started.html
 */

public class UploadFile {
    private static final Logger logger = LoggerFactory.getLogger(UploadFile.class);
    public final String bucketName = "x-" + UUID.randomUUID();
    public final String key = UUID.randomUUID().toString();
    public String filePath;

    public UploadFile() {
        this.setUp();
    }

    public static void main(String[] args) {
        UploadFile upload = new UploadFile();
        upload.uploadFile(S3ClientFactory.transferManager, upload.bucketName, upload.key, upload.filePath);
        upload.cleanUp();
    }

    // snippet-start:[s3.tm.java2.uploadfile.main]
    public String uploadFile(S3TransferManager transferManager, String bucketName,
            String key, String filePath) {
        UploadFileRequest uploadFileRequest = UploadFileRequest.builder()
                .putObjectRequest(b -> b.bucket(bucketName).key(key))
                .addTransferListener(LoggingTransferListener.create())
                .source(Paths.get(filePath))
                .build();

        FileUpload fileUpload = transferManager.uploadFile(uploadFileRequest);

        CompletedFileUpload uploadResult = fileUpload.completionFuture().join();
        return uploadResult.response().eTag();
    }
    // snippet-end:[s3.tm.java2.uploadfile.main]

    private void setUp() {
        S3ClientFactory.s3Client.createBucket(b -> b.bucket(bucketName));
        // get the file system path to the provided file to upload
        URL resource = UploadFile.class.getClassLoader().getResource("image.png");
        filePath = resource.getPath();
    }

    public void cleanUp() {
        S3ClientFactory.s3Client.deleteObject(b -> b.bucket(bucketName).key(key));
        S3ClientFactory.s3Client.deleteBucket(b -> b.bucket(bucketName));
    }
}