AWS Lambda init phase timeout in VPC

263 Views Asked by At

Description

I have a Lambda function (deployed as docker image) that works fine when not in a VPC. The first thing it does is connect to the AWS Secret manager to get credentials and then it uses it to connect to an external web API to get some data. I need to move the Lambda to a VPC to be able to access a MongoDB Atlas database. I've used the following Terraform module to create my VPC:

module "vpc" {
  source = "terraform-aws-modules/vpc/aws"

  name = "vpc-mongodb-atlas"
  cidr = "10.0.0.0/16"

  azs             = ["eu-west-2a", "eu-west-2b"]
  private_subnets = ["10.0.1.0/24", "10.0.2.0/24"]
  public_subnets = ["10.0.0.0/24"]

  enable_nat_gateway = true
  single_nat_gateway = true
}

And then I added my Lambda to this VPC:

module "lambda_function_container_image" {
  source = "terraform-aws-modules/lambda/aws"

...

  image_uri = module.docker_image.image_uri
  package_type = "Image"

...

  vpc_subnet_ids         = module.vpc.private_subnets
  vpc_security_group_ids = [module.vpc.default_security_group_id]
  attach_network_policy              = true
}

Problem

After provisioning the above Terraform my Lambda started timing out during the init phase with the following logs:

INIT_REPORT Init Duration: 10007.38 ms Phase: init Status: timeout

I have a suspicion that it cannot connect to the Internet, but after browsing every possible post and instruction and trying everything I could I cannot really understand what is wrong in this setup and how I could make it work... Any help would be really appreciated.

More info from AWS web interface

VPC configuration: enter image description here enter image description here

Private subnet route table (points to a public NAT gateway): enter image description here Public subnet route table (points to an Internet gateway): enter image description here

Network ACLs: enter image description here enter image description here

Lambda VPC configuration: enter image description here

1

There are 1 best solutions below

0
wikiselev On

It turned out if you use terraform-aws-modules/vpc to provision VPC, by default it sets the default security group to have no inbound and no outbound rules. After allowing all outbound traffic by adding the following settings my Lambda function was able to connect to the Internet:

Original VPC module:

module "vpc" {
  source = "terraform-aws-modules/vpc/aws"

  name = "vpc-mongodb-atlas"
  cidr = "10.0.0.0/16"

  azs             = ["eu-west-2a", "eu-west-2b"]
  private_subnets = ["10.0.1.0/24", "10.0.2.0/24"]
  public_subnets = ["10.0.0.0/24"]

  enable_nat_gateway = true
  single_nat_gateway = true
}

Updated VPC module (with default_security_group settings):

module "vpc" {
  source = "terraform-aws-modules/vpc/aws"

  name = "vpc-mongodb-atlas"
  cidr = "10.0.0.0/16"

  azs             = ["eu-west-2a", "eu-west-2b"]
  private_subnets = ["10.0.1.0/24", "10.0.2.0/24"]
  public_subnets = ["10.0.0.0/24"]

  enable_nat_gateway = true
  single_nat_gateway = true

  # NEW SETTINGS
  manage_default_security_group = true
  default_security_group_egress = [
    {
      rule_no    = 100
      action     = "allow"
      from_port  = 0
      to_port    = 0
      protocol   = "-1"
      cidr_blocks = "0.0.0.0/0"
    },
  ]
}