My use case is that I would like to setup a opentelemetry-collector gateway in a multi-tenant type format. So instead of hardcoding the prometheusremotewrite Authentication header for a single tenant, I was hoping I could use the OTEL_EXPORTER_OTLP_METRICS_HEADERS with each tenant and they would send their credentials as a header. The prometheusremoteagent would then apply this to it's writes to a Cortex backend, storing each tenants metrics into their own tenant space. Is this something that is possible right now?
I have tried setting up an application and otel-collector with the following setups.
# otel-collector-config.yaml
receivers:
# Data sources: traces, metrics, logs
otlp:
protocols:
grpc:
endpoint: 0.0.0.0:4317
http:
endpoint: 0.0.0.0:4318
processors:
# Data sources: traces, metrics, logs
batch:
exporters:
# Data sources: metrics
prometheusremotewrite:
endpoint: "https://<cortex_endpoint>/api/prom"
external_labels:
team: foo
env: bar
service:
pipelines:
metrics:
receivers: [otlp]
processors: [batch]
exporters: [prometheusremotewrite]
For my application, I've added the env variable:
OTEL_EXPORTER_OTLP_METRICS_HEADERS=Authorization=Bearer "<id>:<token>"
This doesn't seem to be working though, at least I'm not seeing metrics populated in my datasource, but I'm not seeing any authentication errors from the prometheusremotewrite exporter either.