Fargate container_definition field "secretOptions" not passsing datadog API_KEY to logConfiguration

Question

I'm trying to send my ECS Fargate logs to Datadog. To do this I need to pass my Datadog API_KEY as a field in the logConfiguration object. I need to secure my API_KEY so I am using AWS Secrets Manager via the secretOptions key of the logConfiguration object.

I'm following the steps from AWS laid out here. The full steps from the Datadog site can be found here

For some reason I dont see the logs show up in datadog. Here is the log config section of my Terraform code under the container_definitions object of the aws_ecs_task_definition resource:

"logConfiguration": {
    "logDriver": "awsfirelens",
    "options": {
        "Name": "datadog",
        "Host": "http-intake.logs.datadoghq.com",
        "dd_service": "myservice",
        "dd_source": "mysource",
        "dd_message_key": "log",
        "dd_tags": "env:dev",
        "TLS": "on",
        "provider": "ecs"
    },
    "secretOptions": [{
        "name": "apikey",
        "valueFrom": "arn:aws:secretsmanager:${data.aws_region.current.name}:${data.aws_caller_identity.current.account_id}:secret:mysecret"
                }]
}

If I take out the secretOptions and add the apikey in plaintext, the logs show up on the datadog console:

"logConfiguration": {
   "logDriver": "awsfirelens",
   "options": {
       "Name": "datadog",
       "Host": "http-intake.logs.datadoghq.com",
       "dd_service": "myservice",
       "dd_source": "mysource",
       "dd_message_key": "log",
       "dd_tags": "env:dev",
       "TLS": "on",
       "provider": "ecs",
       "apikey": "myapikey"
   }
}

I of course cant just send my API_KEY in plaintext. Does the secretOptions just not work for Datadog? Any help is appreciated.

Answer 1

At least the documentation suggests it: "Use a TaskDefinition secret to avoid exposing the apikey in plain text."

Datadog documentation

Works like a charm here. Same configuration used with Terraform 0.15.3 and AWS Provider 3.41.0