aws-sdk-go-v2 PutObject api error AccessDenied

Question

In our Staging environment, we have credential-less access to our private S3 buckets. Access is granted to individual Docker containers. I am trying to upload a file using PutObject using the aws-sdk-go-v2 SDK library, but I'm continually getting a 403 AccessDenied api error.

My upload code looks like this:

var uploadFileFunc = func(s3Details S3Details, key string, payload []byte, params MetadataParams) (*s3.PutObjectOutput, error) {
    client := getS3Client(s3Details)

    return client.PutObject(context.TODO(), &s3.PutObjectInput{
        Bucket:      aws.String(s3Details.Bucket),
        Key:         aws.String(key),
        Body:        bytes.NewReader(payload),
        ContentType: aws.String("text/xml"),
    })
}

func getS3Client(s3Details S3Details) *s3.Client {
    endpointResolver := aws.EndpointResolverWithOptionsFunc(func(service, region string, options ...interface{}) (aws.Endpoint, error) {
        if s3Details.EndpointUrl != "" {
            return aws.Endpoint{
                PartitionID:   "aws",
                URL:           s3Details.EndpointUrl,
                SigningRegion: s3Details.Region,
                SigningMethod: s3Details.SignatureVersion,
            }, nil
        }
        return aws.Endpoint{}, &aws.EndpointNotFoundError{}
    })

    cfg, _ := config.LoadDefaultConfig(context.TODO(),
        config.WithEndpointDiscovery(aws.EndpointDiscoveryEnabled),
        config.WithEndpointResolverWithOptions(endpointResolver))

    return s3.NewFromConfig(cfg, func(o *s3.Options) {
        o.Region = s3Details.Region
        o.Credentials = aws.AnonymousCredentials{}
        o.UsePathStyle = true
    })
}

I am using aws.AnonymousCredentials{} (as our access is credential-less) but this is only to be used for unsigned requests. I cannot use NewStaticCredentialsProvider with empty values for AccessKeyID and/or SecretAccessKey as this will throw a StaticCredentialsEmptyError during the Retrieve(). Adding dummy credentials will throw an error that they are not on record. I am assuming that this is the cause of my 403 AccessDenied.

How do I sign requests without providing credentials in the Go SDK? Is it even possible? In the boto3 Python library this works fine.

Answer 1

First of all, I'll strongly suggest you use the v2 of the AWS SDK of Go. I'll present here how I do this so far.
First, I get the AWS config to use with this code (only relevant parts are shown):

cfg, err := config.LoadDefaultConfig(context.TODO())
    if err != nil {
        Log.Fatal(err)
    }

Here the package used is github.com/aws/aws-sdk-go-v2/config. Then, I instantiate an s3Client to use for contacting AWS S3 service:

s3Client := s3.NewFromConfig(*cfg)

Here, we use this package github.com/aws/aws-sdk-go-v2/service/s3. Finally, to post your object you have to run this code:

input := &s3.PutObjectInput{
    Key:    aws.String("test"),
    Bucket: aws.String("test"),
    Body:   bytes.NewReader([]byte("test")),
    ACL:    types.ObjectCannedACLPrivate,
}

if _, err := s3Client.PutObject(context.TODO(), input); err != nil {
    return "", fmt.Errorf("fn UploadFile %w", err)
}

The new package used here is github.com/aws/aws-sdk-go-v2/service/s3/types.
This code is a simplification but you should able to achieve what you need. Furthermore, it should take very little time to update the version of the SDK and you can rely on both of them simultaneously if you've to work with a huge codebase.
Let me know if this helps!

Edit

I updated my solution by using the aws.AnonymousCredentials{} option. Now I was successfully able to upload a file into an s3 bucket with these options. Below you can find the entire solution:

package main

import (
    "bytes"
    "context"
    "crypto/tls"
    "net/http"

    "github.com/aws/aws-sdk-go-v2/aws"
    "github.com/aws/aws-sdk-go-v2/config"
    "github.com/aws/aws-sdk-go-v2/service/s3"
    "github.com/aws/aws-sdk-go-v2/service/s3/types"
)

func GetAwsConfig() (*aws.Config, error) {
    cfg, err := config.LoadDefaultConfig(context.TODO(),
        // config.WithClientLogMode(aws.LogRequestWithBody|aws.LogResponseWithBody),
        config.WithRegion("eu-west-1"),
        config.WithHTTPClient(&http.Client{Transport: &http.Transport{
            TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
        }}),
        config.WithEndpointResolverWithOptions(
            aws.EndpointResolverWithOptionsFunc(func(service, region string, options ...interface{}) (aws.Endpoint, error) {
                return aws.Endpoint{
                    PartitionID:       "aws",
                    URL:               "http://127.0.0.1:4566",
                    SigningRegion:     "eu-west-1",
                    HostnameImmutable: true,
                }, nil
            }),
        ))
    if err != nil {
        return nil, err
    }
    return &cfg, err
}

func main() {
    cfg, _ := GetAwsConfig()
    s3Client := s3.NewFromConfig(*cfg, func(o *s3.Options) {
        o.Credentials = aws.AnonymousCredentials{}
    })

    if _, err := s3Client.PutObject(context.Background(), &s3.PutObjectInput{
        Bucket: aws.String("mybucket"),
        Key:    aws.String("myfile"),
        Body:   bytes.NewReader([]byte("hello")),
        ACL:    types.ObjectCannedACLPrivate,
    }); err != nil {
        panic(err)
    }
}

Before running the code, you've to create the bucket. I used the below command:
aws --endpoint-url=http://localhost:4566 s3 mb s3://mybucket
Thanks to this you can upload the file into the mybucket s3 bucket. To check for the file existence you can issue this command:
aws --endpoint-url=http://localhost:4566 s3 ls s3://mybucket --recursive --human-readable
Hope this helps in solving your issue!