Using open telemetry collector for many services

Question

From reading documentation at https://opentelemetry.io/docs/collector/ I have a question as it's not clear to me.

We have a number of services(20) running on aws fargate. We are a small company and we want to keep our budget costs down.

We don't want to use aws distro, and we don't want to use side car for each service.

Is an option to run 1 open telemetry collector on an ec2 instance or run as a fargate service and push all services metrics to this one collector? I know that the answer may be "it depends", but in general is there any issue with this approach?

Thanks

Answer 1

You can use the same Collector to receive from multiple applications and process + export them differently by creating multiple pipelines.

For example:

receivers:
  otlp/auth:
    protocols:
      http:
        endpoint: localhost:4318
  otlp/api:
    protocols:
      http:
        endpoint: localhost:4328
exporters:
  otlp:
    endpoint: otelcol:4317
    tls:
      cert_file: file.cert
      key_file: file.key

service:
  extensions: []
  pipelines:
    traces/auth:
      receivers: [otlp/auth]
      processors: []
      exporters: [otlp]
    traces/api:
      receivers: [otlp/api]
      processors: []
      exporters: [otlp]
# (Equivalent to:)
#    traces:
#      receivers: [otlp/auth, otlp/api]
#      processors: []
#      exporters: [otlp]

See the talk by Juraci Paixão Kröhling on OpenTelemetry Collector Deployment Patterns to see the many different ways the Collector can be deployed.

The diagram above is from a talk by Dan Jaglowski on Connected Observability Pipelines in the OpenTelemetry Collector, showing how the Collector can receive signals from two different receivers, each going through different processing, and converge to export it to a single backend.

The main concern with having just one instance of the Collector is its ability to handle signals at scale. If you have too many applications sending in too many signals, the Collector may not have enough system resources to handle them. So it's important to monitor the Collector itself to see if Spans are being refused or the exporter queue is getting too long. But 20 services sounds like something that can be handled by a single Collector. See Scaling the Collector for more information.