X-Ray SDK tracing support for Quarkus Netty (Vert.x)

56 Views Asked by At

I'd like to ask if there is any way to use tracing incoming requests with the X-Ray SDK for quarkus reactive (Netty) application?

To be more specific, I am trying to trace all hits to dynamoDB.

Following AWS documentation (https://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-java-filters.html) I can see solutions only for tomcat/spring.

Example solution is to provide bean for javax.servlet.Filter:

@Configuration
public class WebConfig {

  @Bean
  public Filter TracingFilter() {
    return new AWSXRayServletFilter(SegmentNamingStrategy.dynamic("MyApp", "*.example.com"));
  }
}

Since Netty doesn't relay on servlets, is there any overcome available?

What I've done so far is adding:

  1. dependencies:
implementation("io.quarkus:quarkus-opentelemetry-exporter-otlp")
implementation("io.opentelemetry:opentelemetry-extension-aws")
implementation("io.opentelemetry.contrib:opentelemetry-aws-xray:1.+")
implementation("io.opentelemetry.contrib:opentelemetry-aws-xray-propagator:1.+")
implementation("com.amazonaws:aws-xray-recorder-sdk-aws-sdk-v2:2.8.0")
  1. properties:
quarkus.opentelemetry.enabled=true
quarkus.opentelemetry.propagators=xray
quarkus.dynamodb.telemetry.enabled=true
quarkus.dynamodb.interceptors=com.amazonaws.xray.interceptors.TracingInterceptor

  1. config-code for idGenerator:
import io.opentelemetry.contrib.awsxray.AwsXrayIdGenerator
import io.opentelemetry.sdk.trace.IdGenerator
import jakarta.enterprise.context.ApplicationScoped

class AwsOpenTelemetryIdGeneratorProducer {

    @ApplicationScoped
    fun idGenerator(): IdGenerator {
        return AwsXrayIdGenerator.getInstance()
    }
}
  1. aws-otel-collector definition in terraform for aws_ecs_task_definition:
{
      name : "aws-otel-collector",
      image : "public.ecr.aws/aws-observability/aws-otel-collector:latest",
      cpu : 0,
      portMappings : [],
      essential : true,
      command : [
        "--config=/etc/ecs/ecs-xray.yaml"
      ],
      environment : [],
      mountPoints : [],
      volumesFrom : [],
      logConfiguration : {
        logDriver : "awslogs",
        options : {
          awslogs-create-group : "true",
          awslogs-group : "/ecs/ecs-aws-otel-sidecar-collector",
          awslogs-region : "eu-central-1",
          awslogs-stream-prefix : "ecs"
        }
      }
    }
  1. IAM policy:
statement {
    actions = [
      "xray:PutTraceSegments",
      "xray:PutTelemetryRecords"
    ]
    resources = ["*"]
  }

With this configuration I am able to trace requests for my ECS. The problem is I cannot trace any dynamoDb hits as following exception occurs "Suppressing AWS X-Ray context missing exception (SegmentNotFoundException): Failed to begin subsegment named 'DynamoDb': segment cannot be found.".

Exception's message is quite meaningful, it cannot determine any parent-segment for DynamoDb sub-segment, what is understandable.

So I decided to add following vertex.core.Handler:

import com.amazonaws.xray.AWSXRay
import com.amazonaws.xray.contexts.SegmentContextExecutors
import com.amazonaws.xray.entities.Segment
import io.quarkus.runtime.annotations.RegisterForReflection
import io.vertx.core.Handler
import io.vertx.ext.web.RoutingContext

@RegisterForReflection
class XRayTracingFilter : Handler<RoutingContext> {

    override fun handle(context: RoutingContext) {
        val request = context.request()
        val segment: Segment = AWSXRay.beginSegment("TestSegment")
        segment.putHttp("request", request.method())
        SegmentContextExecutors.newSegmentContextExecutor(segment)
        context.addEndHandler { segment.close() }
        context.next()
    }
}

which then I register as follows:

import io.quarkus.vertx.http.runtime.filters.Filters
import jakarta.enterprise.context.ApplicationScoped
import jakarta.enterprise.event.Observes

class XrayConfig {

    @ApplicationScoped
    fun registerFilters(@Observes filters: Filters) {
        filters.register(XRayTracingFilter(), 1)
    }
}

After adding XRayTracingFilter exception SegmentNotFoundException doesn't occur anymore but unfortunately I'm still unable to see any tracing coming from AWS::DynamoDB::Table node. All I still see comes from AWS::ECS::Fargate.

I cannot understand at this point why the problem still occurs and which part of my configuration it is related to. Is it a problem with Otel-collector, my Handler or IAM policy?

STACK INFO:

  • Quarkus 3.6.6
  • Java 17 Corretto
  • Gradle 8.5

I would be super glad for any feedback/ideas for my issue.

Thank you in advance!

0

There are 0 best solutions below