I have a yaml file which is similar to the following (FYI: ssm_secrets can be an empty array):
rabbitmq:
repo_name: bitnami
namespace: rabbitmq
target_revision: 11.1.1
path: rabbitmq
values_file: charts/rabbitmq/values.yaml
ssm_secrets: []
app_name_1:
repo_name: repo_name_1
namespace: namespace_1
target_revision: target_revision_1
path: charts/path
values_file: values.yaml
ssm_secrets:
- name: name-dev-1
key: .env
ssm_path: ssm_path/dev
name-backend:
repo_name: repo_name_2
namespace: namespace_2
target_revision: target_revision_2
path: charts/name-backend
values_file: values.yaml
ssm_secrets:
- name: name-backend-app-dev
ssm_path: name-backend/app/dev
key: app.ini
- name: name-backend-abi-dev
ssm_path: name-backend/abi/dev
key: contractTokenABI.json
- name: name-backend-widget-dev
ssm_path: name-backend/widget/dev
key: name.ini
- name: name-abi-dev
ssm_path: name-abi/dev
key: name_1.json
- name: name-website-dev
ssm_path: name/website/dev
key: website.ini
- name: name-name-dev
ssm_path: name/name/dev
key: contract.ini
- name: name-key-dev
ssm_path: name-key/dev
key: name.pub
And using External Secrets and EKS Blueprints, I am trying to generate the yaml file necessary to create the secrets
resource "kubectl_manifest" "secret" {
for_each = toset(flatten([for service in var.secrets : service.ssm_secrets[*].ssm_path]))
yaml_body = <<YAML
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: ${replace(each.value, "/", "-")}
namespace: ${split("/", each.value)[0]}
spec:
refreshInterval: 30m
secretStoreRef:
name: ${local.cluster_secretstore_name}
kind: ClusterSecretStore
data:
- secretKey: .env
remoteRef:
key: ${each.value}
YAML
depends_on = [kubectl_manifest.cluster_secretstore, kubernetes_namespace_v1.namespaces]
}
The above works fine, but I also need to use the key value from the yaml into secretKey: <key_value from yaml>.
If I try with for_each = toset(flatten([for service in var.secrets : service.ssm_secrets[*]]))
resource "kubectl_manifest" "secret" {
for_each = toset(flatten([for service in var.secrets : service.ssm_secrets[*]]))
yaml_body = <<YAML
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: ${replace(each.value["ssm_path"], "/", "-")}
namespace: ${split("/", each.value["ssm_path"])[0]}
spec:
refreshInterval: 30m
secretStoreRef:
name: ${local.cluster_secretstore_name}
kind: ClusterSecretStore
data:
- secretKey: .env
remoteRef:
key: ${each.value["ssm_path"]}
YAML
depends_on = [kubectl_manifest.cluster_secretstore, kubernetes_namespace_v1.namespaces]
}
It just gives me the following error:
The given "for_each" argument value is unsuitable: "for_each" supports maps and sets of strings, but you have provided a set containing type object.
I have tried converting the variable into a map, used lookup, but it doesn't work. Any help would be much appreciated.
Update 1:
As per @MattSchuchard suggestion, changing the for_each into
for_each = toset(flatten([for service in var.secrets : service.ssm_secrets]))
Gave the following error:
Error: Invalid for_each set argument
│
│ on ../../modules/02-plugins/external-secrets.tf line 58, in resource "kubectl_manifest" "secret":
│ 58: for_each = toset(flatten([for service in var.secrets : service.ssm_secrets]))
│ ├────────────────
│ │ var.secrets is object with 14 attributes
│
│ The given "for_each" argument value is unsuitable: "for_each" supports maps and sets of strings, but you have provided a set containing type object.
Update 2:
@mariux gave the perfect solution, but here is what I came up with. It's not that cleaner, but definitely works (PS: I myself am going to use Mariux's solution):
locals {
my_list = tolist(flatten([for service in var.secrets : service.ssm_secrets[*]]))
}
resource "kubectl_manifest" "secret" {
count = length(local.my_list)
yaml_body = <<YAML
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
name: ${replace(local.my_list[count.index]["ssm_path"], "/", "-")}
namespace: ${split("/", local.my_list[count.index]["ssm_path"])[0]}
spec:
refreshInterval: 30m
secretStoreRef:
name: ${local.cluster_secretstore_name}
kind: ClusterSecretStore
data:
- secretKey: ${local.my_list[count.index]["key"]}
remoteRef:
key: ${local.my_list[count.index]["ssm_path"]}
YAML
depends_on = [kubectl_manifest.cluster_secretstore, kubernetes_namespace_v1.namespaces]
}
Assumptions
Based on what you shared, i make the following assumptions:
ssm_secrets.*.name
using the givenkey
andssm_path
attributes.name
is globally unique for all services and never reused.terraform hacks
Based on the assumptions you can create an array of ALL ssm_secrets using
and convert it to a map that can be used in
for_each
by keying the values by.name
:Full (working) example
The example below works for me and makes some assumption where the variables should be used.
yamldecode
to decode your original input intolocal.input
yamlencode
to make reading the manifest easier and removing some string interpolcations. This also ensures that the indent is correct as we convert HCL to yaml.A
terraform init && terraform plan
will plan to create the following resources:hint: you could also try to use the
kubernetes_manifest
resource instead ofkubectl_manifest
p.s.: We created Terramate to make complex creation of Terraform code easier. But this seems perfectly fine for pure Terraform.