ResourceInitializationError: failed to validate logger args: : signal: killed

Question

Suddenly getting the message " ResourceInitializationError: failed to validate logger args: : signal: killed" while starting AWS ECS Fargate Service. Same service was running fine couple of days back.

Following is log driver configurations in related aws task:

Log Configuration
Log driver: awslogs
Key Value
awslogs-group /ecs/analytics
awslogs-region us-east-1
awslogs-stream-prefix ecs

Any idea or help?

Answer 1

Came across this issue today. The issue was that the log group I specified didn't exist yet. If you don't want to manually create it, make sure to add the awslogs-create-group and set it to "true". You'll have to grant your ECS Task Execution role a logs:CreateLogGroup permission as well.

  "logConfiguration": {
    "logDriver": "awslogs",
    "secretOptions": null,
    "options": {
      "awslogs-create-group": "true",
      "awslogs-group": "/ecs/app",
      "awslogs-region": "ap-southeast-2",
      "awslogs-stream-prefix": "ecs"
    }
  }

Reference: https://docs.aws.amazon.com/AmazonECS/latest/developerguide/using_awslogs.html

Answer 2

I finally found the root cause:

The error appears if the fargate service is not able to connect to the CloudWatch api endpoint. This might happen if you have fargate running in a private subnet without internet access. You could either add the CloudWatch log Endpoint to your private subnet or add internet connectivity

Answer 3

Adding CloudWatchLogsFullAccess policy to ECS task execution role solved my problem (suggested by @redgeoff )

Answer 4

I recently spent hours on this same issue. It turns out that the log group and stream prefix specified in my container definition didn't exist.

It would be wonderful if AWS could provide helpful error messages...

Answer 5

I got this error, checked my NAT and IG, and all is good. And I found the endpoint interface also was set up as com.amazonaws.use-ease-1.logs Nothing seems to need to change. Finally, I deleted the interface endpoint and the error went away. But I am still confusing what happened.

Answer 6

I just experienced this. I have ECS Fargate running and I've just added a VPC endpoint for Cloudwatch Logs com.amazonaws.REGION.logs in my account. When I added the VPC endpoint my logs stopped appearing.

In order to remedy this without deleting the VPC endpoint again, for my setup with Fargate running with internet access I had to ensure that:

1. My ECS service had a security group rule that to allows HTTPS traffic outbound

   {
      type: egress
      port_to: 443   
      port_from: 443
      protocol: TCP
   }
   

2. That my new VPC Endpoint had a security group rule to allow HTTPS traffic inbound from my ECS security group

   {
      type: ingress
      port_to: 443   
      port_from: 443
      protocol: TCP
      source_security_group_id: [Your ECS SECURITY GROUP ID]
   }