Using smart-open with AWS lambda S3 access error

394 Views Asked by At

I'm trying to stream a file from my requests.get to my s3 using smart-open within my lambda. My lambda role has proper access to the s3, needed action of "s3:GetObject","s3:PutObject", and kms key to decrypted bucket.

I'm getting error message the bucket 'my-bucket-xyz' does not exist, or is forbidden for access (ClientError('An error occurred (AccessDenied) when calling the CreateMultipartUpload operation: Access Denied'

My hunch is something is wrong with the way I'm using smart-open, but not sure and just starting using smart-open today.

My code:

def download_report(baseURL):

    bucket = "my-bucket-xyz"

    response = requests.get(baseURL, stream=True)
    s3url = "s3://" + bucket + '/reports.txt'
    with open(s3url, 'w', transport_params = {'client_kwargs': {'S3.Client.create_multipart_upload': { 'ServerSideEncryption': 'aws:kms'}},'client': boto3.client('s3')}) as fout:
        
       fout.write(response.content)

    return
0

There are 0 best solutions below