I am trying to retrieve files from a bucket in S3 through my code in AWS lambda. The same code will then retrieve and insert the appropriate data from the file in mongoDB, hosted on EC2.
Problem encountered: I cannot read the objects from the bucket. The instance just timed out on the log console.
For the VPC EndPoints, Ive allowed Full Access as policy, modified the route table(Deleted the route to the Internet gateway) and changed the security group that restricted the outbound traffic(Type list HTTPS and HTTP and destination as prefix list ID for the endpoint service)
As for the IAM roles, i've set the following policies : AmazonS3FullAccess , AdministratorAccess and AWSLambdaExecute .
In addition, I've added the following inline policy to restrict access through the specific vpc endpoints
{
"Version": "2012-10-17",
"Statement": [
{
"Action": "s3:*",
"Effect": "Deny",
"Resource": [
"arn:aws:s3:::my_secure_bucket",
"arn:aws:s3:::my_secure_bucket/*"
],
"Condition": {
"StringNotEquals": {
"aws:sourceVpce": "vpce-1a2b3c4d"
}
}
}
]
}
Both S3 and EC2 are in the US-WEST-2 region. Here's part of my lambda code that deals with S3:
AmazonS3Client s3Client = new AmazonS3Client(new DefaultAWSCredentialsProviderChain());
for (S3EventNotificationRecord record : s3event.getRecords()) {
String srcKey = record.getS3().getObject().getKey().replace('+', ' ');
srcKey = URLDecoder.decode(srcKey, "UTF-8");
String srcBucket = record.getS3().getBucket().getName();
context.getLogger().log("found id: " + srcBucket+" "+srcKey);
// retrieve s3 object
S3Object object = s3Client.getObject(new GetObjectRequest(srcBucket, srcKey));
//My function times out while performing the above line!
InputStream objectData = object.getObjectContent();
}