AWS X-RAY [Error] Sending segment batch failed with: NoCredentialProviders: no valid providers in chain. Deprecated

Question

I'm using X-Ray Daemon with Fargate. Unfortunately in the logs of the daemon I see errors like:

[Error] Sending segment batch failed with: NoCredentialProviders: no valid providers in chain. Deprecated. For verbose messaging see aws.Config.CredentialsChainVerboseErrors Warn] Delaying sending of additional batches by 0 seconds

AWS X-Ray works with IAM: permission: AWSXrayFullAccess

I already checked:

• https://github.com/aws-samples/aws-xray-fargate
• https://docs.amazonaws.cn/en_us/xray/latest/devguide/xray-guide.pdf

{
 "name" : "xray-daemon",
 "image" : "amazon/aws-xray-daemon",
 "user" : "1337",
 "essential" : true,
 "cpu" : "32",
 "memoryReservation" : "256",
 "portMappings" : [
 {
 "containerPort" : 2000,
 "protocol" : "udp"
 }
 ]
 }
 ],
 "requiresCompatibilities" : [ "FARGATE" ],
 "taskRoleArn" : "arn:aws:iam::123456789012:role/ecsTaskRole",
 "executionRoleArn" : "arn:aws:iam::123456789012:role/ecsTaskExecutionRole",
 "networkMode" : "awsvpc"
} 

Answer 1

I got this error when my version of the x-ray deamon was old.

Follow this as guidance to update your x-ray deamon if it is running on ECS:

https://docs.aws.amazon.com/xray/latest/devguide/xray-daemon-ecs.html

Answer 2

You should check you role permissions role, I used the same for execution & taskRoleArn. my role contained these policies

• arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly
• arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy
• arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy
• arn:aws:iam::aws:policy/AWSXRayFullAccess

and also check trust relationships for role. my used:

• ecs-tasks.amazonaws.com
• ecs.amazonaws.com
• c2.amazonaws.com

Answer 3

XRay daemon is missing credentials needed to authenticate with XRay server. You can pass credentials to XRay daemon in several ways, but the most common are:

1. through environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
2. through ~/.aws/credentials file (see https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-files.html)

I'm guessing option 1 will be easier with fargate.

Of course you need to generate the key in IAM, see: https://docs.aws.amazon.com/powershell/latest/userguide/pstools-appendix-sign-up.html

The user you'll create the access key for will need to have permissions to use XRay (AWSXrayFullAccess should do).

Answer 4

option 1: As Lukasz said, All you want to do is to pass environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY) during container creation.

and about execution role you shouldn't worry about it, recently AWS added the below policies by default to task execution role

• AWSXRayDaemonWriteAccess
• AmazonECSTaskExecutionRolePolicy
• systems-manager-parameters

once you get [Info] Successfully sent batch of 1 segments (0.003 seconds) log in your task cloud watch just check the service map tab in X-Ray and you will find the graph

Option 2: you may try using the old version (amazon/aws-xray-daemon:1)