X-Ray Daemon don't receive any data from envoy

Question

I have a service running a task definition with three containers:

1. service itself
2. envoy
3. x-ray daemon

And I want to trace and monitor my services interacting with each other with x-ray. But I don't see any data in x-ray.

I can see the request logs and everything in the envoy logs but there are no error messages about missing connection to the x-ray daemon. Envoy container has three env variables:

APPMESH_VIRTUAL_NODE_NAME = mesh/mesh-name/virtualNode/service-virtual-node
ENABLE_ENVOY_XRAY_TRACING = 1
ENVOY_LOG_LEVEL = trace

The x-ray daemon is pretty plain and has just a name and an image (amazon/aws-xray-daemon:1). But when looking in the logs of the x-ray dameon, there is only the following:

2022-05-31T14:48:05.042+02:00   2022-05-31T12:48:05Z [Info] Initializing AWS X-Ray daemon 3.0.0
2022-05-31T14:48:05.042+02:00   2022-05-31T12:48:05Z [Info] Using buffer memory limit of 76 MB
2022-05-31T14:48:05.042+02:00   2022-05-31T12:48:05Z [Info] 1216 segment buffers allocated
2022-05-31T14:48:05.051+02:00   2022-05-31T12:48:05Z [Info] Using region: eu-central-1
2022-05-31T14:48:05.788+02:00   2022-05-31T12:48:05Z [Error] Get instance id metadata failed: RequestError: send request failed
2022-05-31T14:48:05.788+02:00   caused by: Get http://169.254.169.254/latest/meta-data/instance-id: dial tcp xxx.xxx.xxx.254:80: connect: invalid argument
2022-05-31T14:48:05.789+02:00   2022-05-31T12:48:05Z [Info] Starting proxy http server on 127.0.0.1:2000

As far as I read, the error you can see in these logs doesn't affect the functionality (https://repost.aws/questions/QUr6JJxyeLRUK5M4tadg944w).

I'm pretty sure I'm missing a configuration or access right. It's running already on staging but I set this up several weeks ago and I don't find any differences between the configurations.

Thanks in advance!

Answer 1

In my case, I made a copy-paste mistake by copying trailing line break into the name of the environment variable ENABLE_ENVOY_XRAY_TRACING which wasn't visible in the overview and only inside the text field.