Spring Boot micrometer custom metrics not exported via HTTP OTEL push into Datadog

167 Views Asked by At

I am using micrometer-registry-otlp, within a spring boot 3.1.6 project. I can see all the metrics appear within the /actuator/metrics endpoint (including the spring default metrics). however, I cannot see the custom metrics (app specific ones) after the metrics has been published, we have OTEL collector and Datadog Exporter in place. Only the spring metrics are exported and available in the Datadog 'metrics explorer'. Any thoughts on what is missing?

spring configs:

# disable all registries unless stated otherwise by the registry-specific property
management.defaults.metrics.export.enabled=false
## Open Telemetry specific
management.otlp.metrics.export.enabled=true
# OTEL_EXPORTER_OTLP_HTTP_ENDPOINT is set into the pod as env. variable, fallback to localhost when not available
management.otlp.metrics.export.url=${OTEL_EXPORTER_OTLP_HTTP_ENDPOINT:http://localhost:4318}/v1/metrics
management.otlp.metrics.export.resource-attributes.service.name=${spring.application.name}
management.metrics.tags.service.name=${spring.application.name}
management.metrics.tags.environment=${ENVIRONMENT_NAME} 

metrics:

[ root@curl-test:/ ]$ curl http://storage-service:8080/actuator/metrics

{"names":["application.started.time","disk.free","disk.total","executor.active","executor.completed","executor.pool.core","executor.pool.max","executor.pool.size","executor.queue.remaining","executor.queued","http.server.requests","http.server.requests.active","jvm.buffer.count","jvm.buffer.memory.used",
............. "storage.download.artifacts","storage.download.artifacts.active","storage.get.artifacts","system.cpu.count","system.cpu.usage","system.load.average.1m","tomcat.sessions.active.current","tomcat.sessions.active.max","tomcat.se

[ root@curl-test:/ ]

missing in Datadog : these metrics are missing in the Metrics Explorer.

"storage.download.artifacts","storage.download.artifacts.active","storage.get.artifacts"

Infra:

The metrics are pushed into an OTEL collector, further the Datadog exporter pushes it into the Datadog. refer: https://docs.datadoghq.com/opentelemetry/collector_exporter/?tab=onahost

Is there any config in Datadog that we are missing? Is the Datadog Exporter filtering unknown attributes? Does it support only predefined integrations?

Thanks in advance.

0

There are 0 best solutions below