In our application we are using API gateway, LambdaFunction and S3 bucket. We are using a presigned_url generated by lambda function to upload videos to a s3 bucket. Below is the code that we are using in lambda function to create the presigned_url.
presigned_url = s3_client.generate_presigned_url(
'put_object',
Params={
'Bucket': bucket_name,
'Key': object_key
}
In us-east-1, video upload is working immediately after creation of the resources. In us-east-2, we tried to upload video immediately after the creation of the resources, but it didn't work. When we tested the same resource after one day, it worked. In us-west-1, we tried to upload video immediately after the creation of the resources, but it didn't work. When we tested the same resource after one day, it worked.
I need to create a working setup in us-east-2, which should work same as us-east-1.
A temporary redirect is a type of error response that signals to the requester that they should resend the request to a different endpoint. Due to the distributed nature of Amazon S3, requests can be temporarily routed to the wrong facility. This is most likely to occur immediately after buckets are created or deleted.
For example, if you create a new bucket and immediately make a request to the bucket, you might receive a temporary redirect, depending on the location constraint of the bucket. If you created the bucket in the US East (N. Virginia) AWS Region, you will not see the redirect because this is also the default Amazon S3 endpoint.
However, if the bucket is created in any other Region, any requests for the bucket go to the default endpoint while the bucket's DNS entry is propagated. The default endpoint redirects the request to the correct endpoint with an HTTP 302 response. Temporary redirects contain a URI to the correct facility, which you can use to immediately resend the request.
https://docs.aws.amazon.com/AmazonS3/latest/dev/Redirects.html
Note that the last part -- which you can use to immediately resend the request -- is not quite accurate. You can -- but if the request uses Signature Version 4, then following the redirect to the new hostname will result in a SignatureDoesNotMatch error because the hostname will be different. Back in the old days of Signature Version 2, the bucket name was included in the signature but the endpoint hostname itself was not, so the redirect to a different endpoint hostname would not invalidate the signature.
None of this would be a problem if boto were doing the right thing and using the correct regional endpoint to create the signed URL -- but for some reason, it uses the "global" (generic) endpoint -- which causes S3 to issue those redirects for the first few minutes of the bucket's lifetime, because DNS hasn't been updated, so the request misroutes to us-east-1 and gets redirected. That's why I suspected us-east-1 wouldn't exhibit the behavior.
This should be the default behavior, but it isn't; still, it seems like there should be a cleaner way to do this, automatically via configuration... and there may be... but I haven't found it in the documentation.
As a workaround, the client constructor accepts an endpoint_url argument, which seems to serve the purpose. As it turns out, s3.${region}.amazonaws.com is a valid endpoint for each S3 region, so they can be constructed from a region string.
s3_client = boto3.client('s3', region_name=region, endpoint_url=('https://s3.' + region + '.amazonaws.com'), config=...)
Long-time users of S3 may be suspicious of the claim that all regions support this, but it is accurate as of this writing. Originally, some regions formerly used a dash rather than a dot, e.g. s3-us-west-2.amazonaws.com and this is still valid in those older regions, but all regions now support the canonical form mentioned above.