I'm trying to stream a file from my requests.get
to my s3 using smart-open
within my lambda. My lambda role has proper access to the s3, needed action of "s3:GetObject","s3:PutObject"
, and kms key to decrypted bucket.
I'm getting error message the bucket 'my-bucket-xyz' does not exist, or is forbidden for access (ClientError('An error occurred (AccessDenied) when calling the CreateMultipartUpload operation: Access Denied'
My hunch is something is wrong with the way I'm using smart-open, but not sure and just starting using smart-open today.
My code:
def download_report(baseURL):
bucket = "my-bucket-xyz"
response = requests.get(baseURL, stream=True)
s3url = "s3://" + bucket + '/reports.txt'
with open(s3url, 'w', transport_params = {'client_kwargs': {'S3.Client.create_multipart_upload': { 'ServerSideEncryption': 'aws:kms'}},'client': boto3.client('s3')}) as fout:
fout.write(response.content)
return