No Traces reported by GCP Load Balancer to Google Cloud Trace

Question

I'm running several applications on GKE with a Spring Cloud Gateway in front of them and behind a Google Cloud Loadbalancer.

While debuging i noticed that most traces reported to Google Cloud Trace show a missing root trace. It appears the Gateway (correctly) reads the X-Cloud-Trace-Context header and reports it own span with a parent traceId. But this parent does not actually exist in trace-id.

i'm using google own spring-integration for tracing and can't find any span reported by a load-balancer, so i'm inclined to believe there is something wrong there.

Do i have to activate tracing on the ingress/load-balancer somewhere? I cant find any documentation on that.

Answer 1

You can consider using a distributed tracing solution. For example like Zipkin or Jaeger, these tools help track requests across microservice architecture. After choosing the tool, integrate the tracing solution's libraries into your application code. This enables capturing trace context and propagating it across service calls.

One approach as well is to activate tracing on the ingress/load-balancer in your Google Kubernetes Engine(GKE) setup, you need to configure the Load Balancer to trace HTTP requests.

Enabling logging on an existing backend service

1. In the Google Cloud console, go to the Load balancing page.

2. Click the name of your load balancer.

3. Click editEdit.

4. Click Backend Configuration.

5. Click editEdit next to your backend service.

6. Click Advanced configurations (Session affinity, connection draining timeout).

7. Click Enable logging.

8. Set a Sample rate fraction. You can set a number from 0.0 through 1.0, where 0.0 means that no requests are logged and 1.0 means that 100% of the requests are logged. The default value is 1.0.

9. Optional: To include all the optional fields in the logs, in the Optional fields section, click Include all optional fields.

10. To finish editing the backend service, click Update.

11. To finish editing the load balancer, click Update.