Terragrunt cannot get provider from generated provider.tf from parent terragrunt.hcl

Question

Project root
├── terraform
│   └── non_live
│       ├── base
│       │   └── terragrunt.hcl
│       ├── global.hcl
│       └── terragrunt.hcl

I have the above directory structure. The parent terragrunt.hcl file is as below

terragrunt_version_constraint = "< v0.52.0"
terraform_version_constraint  = ">= 1.5.5, < 1.6.0"

remote_state {
  backend = "s3"
  generate = {
    path      = "backend.tf"
    if_exists = "overwrite"
  }
  config = {
    bucket         = "my-bucket"
    key            = "${path_relative_to_include()}/terraform.tfstate"
    region         = "us-west-1"
    encrypt        = true
    dynamodb_table = "my-lock-table"
  }
}

generate "provider" {
  path      = "provider.tf"
  if_exists = "overwrite_terragrunt"
  contents  = <<EOF
provider "aws" {
  region  = "us-west-1"
  profile = "my-profile"
}
EOF
}

generate "version" {
  path      = "terraform.tf"
  if_exists = "overwrite_terragrunt"
  contents  = <<EOF
terraform {
  required_version = "1.5.5"
}
EOF
}

The child terragrunt.hcl file inside non_live/base/terragrunt.hcl file is as below

include "root" {
  path = find_in_parent_folders()
}

include "global" {
  path   = "${get_terragrunt_dir()}/../global.hcl"
  expose = true
}

terraform {
  source = "tfr:///terraform-aws-modules/s3-bucket/aws//.?version=3.15.1"
}

inputs = {
  bucket                                = "terragrunt-test-bucket"
  attach_deny_insecure_transport_policy = true
  s3_bucket_region                      = "${include.global.locals.region}"
}

When I try to initialize terragrunt inside non_live/base/ directory. The provider.tf file gets generated inside .terragrunt-cache directory but it was suppose to generated inside non_live/base/provider.tf as far my understanding. However, it seems like when I try to init terragrunt, it is not able to find the AWS profile and give the following error.

ERRO[0007] Error finding AWS credentials (did you set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environment variables?): NoCredentialProviders: no valid providers in chain. Deprecated.
        For verbose messaging see aws.Config.CredentialsChainVerboseErrors 
ERRO[0007] Unable to determine underlying exit code, so Terragrunt will exit with error code 1 

Can anyone please explain why terragrunt is not able to find the AWS provider and how can I make terragrunt able to find the AWS provider config. FYI, when I set AWS env variable export AWS_PROFILE=my-profile then it works fine.

Answer 1

After some debugging, I found the answer of the problem. I did not set any profile in the backend config section. So terragrunt was not able to understand which profile/ AWS credentials to use to connect with the S3 bucket. The following backend configuration solved the issue for me

remote_state {
  backend = "s3"
  generate = {
    path      = "backend.tf"
    if_exists = "overwrite"
  }
  config = {
    bucket         = "my-bucket"
    key            = "${path_relative_to_include()}/terraform.tfstate"
    region         = "us-west-1"
    encrypt        = true
    dynamodb_table = "my-lock-table"
    profile        = "my-profile"
  }
}