Placing file inside folder of S3 bucket

Question

have a spring boot application, where I am tring to place a file inside folder of S3 target bucket. target-bucket/targetsystem-folder/file.csv

The targetsystem-folder name will differ for each file which will be retrived from yml configuration file.

The targetsystem-folder have to created via code if the folder doesnot exit and file should be placed under the folder

As I know, there is no folder concept in S3 bucket and all are stored as objects.

Have read in some documents like to place the file under folder, have to give the key-expression like targetsystem-folder/file.csv and bucket = target-bucket. But it doesnot work out.Would like to achieve this using spring-integration-aws without using aws-sdk directly

    <int-aws:s3-outbound-channel-adapter id="filesS3Mover"
        channel="filesS3MoverChannel"
        transfer-manager="transferManager"
        bucket="${aws.s3.target.bucket}"
        key-expression="headers.targetsystem-folder/headers.file_name"
        command="UPLOAD">
</int-aws:s3-outbound-channel-adapter>

Can anyone guide on this issue

Answer 1

Your problem that the SpEL in the key-expression is wrong. Just try to start from the regular Java code and imagine how you would like to build such a value. Then you'll figure out that you are missing concatenation operation in your expression:

 key-expression="headers.targetsystem-folder + '/' + headers.file_name"

Also, please, in the future provide more info about error. In most cases the stack trace is fully helpful.

Answer 2

In the project that I was working before, I just used the java aws sdk provided. Then in my implementation, I did something like this

private void uploadFileTos3bucket(String fileName, File file) {
    s3client.putObject(new PutObjectRequest("target-bucket", "/targetsystem-folder/"+fileName, file)
            .withCannedAcl(CannedAccessControlList.PublicRead));
}

I didn't create anymore configuration. It automatically creates /targetsystem-folder inside the bucket(then put the file inside of it), if it's not existing, else, put the file inside.

---

You can take this answer as reference, for further explanation of the subject.

There are no "sub-directories" in S3. There are buckets and there are keys within buckets.

You can emulate traditional directories by using prefix searches. For example, you can store the following keys in a bucket:

foo/bar1
foo/bar2
foo/bar3
blah/baz1
blah/baz2