I'm following Istio + Loki tutorial everything works fine but then I've tried to add Grafana Tempo to receive the traces and it's not working, have tried different configuration and realised the 'TracerExporter' is not showing in the logs, I guess I'm doing something wrong in the opentelemetry-collector-contrib configMap, appreciate any help, following configuration.
ConfigMap - opentelemetry-collector-conf
apiVersion: v1
data:
opentelemetry-collector-config: |
receivers:
otlp:
protocols:
grpc:
http:
processors:
batch:
attributes:
actions:
- action: insert
key: loki.attribute.labels
value: pod, namespace,cluster,mesh,hostname
exporters:
loki:
endpoint: "http://loki.istio-system.svc:3100/loki/api/v1/push"
logging:
loglevel: debug
jaeger:
endpoint: jaeger-collector.istio-system.svc:14250
tls:
insecure: true
otlp:
endpoint: tempo.istio-system.svc:4317
tls:
insecure: true
prometheusremotewrite:
endpoint: http://prometheus.istio-system.svc.cluster.local:9090
target_info:
enabled: false
extensions:
health_check:
service:
extensions:
- health_check
pipelines:
metrics:
receivers: [otlp]
processors: [batch]
exporters: [prometheusremotewrite]
traces:
receivers: [otlp]
processors: []
exporters: [jaeger, otlp]
logs:
receivers: [otlp]
processors: [attributes, batch]
exporters: [loki, logging]
kind: ConfigMap
kubectl logs -l app=opentelemetry-collector -n istio-system -f
2023-10-21T21:46:53.525Z info LogsExporter {"kind": "exporter", "data_type": "logs", "name": "logging", "#logs": 1}
2023-10-21T21:46:53.525Z info ResourceLog #0
Resource SchemaURL:
ScopeLogs #0
ScopeLogs SchemaURL:
InstrumentationScope
LogRecord #0
ObservedTimestamp: 1970-01-01 00:00:00 +0000 UTC
Timestamp: 2023-10-21 21:46:52.845122 +0000 UTC
SeverityText:
SeverityNumber: Unspecified(0)
Body: Str([2023-10-21T21:46:52.845Z] "GET /api/people HTTP/1.1" 401 - via_upstream - "-" 497 175 15 11 "10.42.0.1" "PostmanRuntime/7.32.3" "5041b7e0-5825-9bd3-8e53-fabe997464ef" "spendingbetter.com" "10.42.2.189:8082" inbound|8082|| 127.0.0.6:35567 10.42.2.189:8082 10.42.0.1:0 - default
)
Attributes:
-> cluster: Str(Kubernetes)
-> hostname: Str(person-service-795ffff75c-nxpgp)
-> mesh: Str(cluster.local)
-> namespace: Str(default)
-> pod: Str(person-service-795ffff75c-nxpgp)
-> status: Str(401)
-> traceId: Str(35b97f83f739fd05c1908f2e134b2ae3)
-> loki.attribute.labels: Str(pod, namespace,cluster,mesh,hostname)
Trace ID:
Span ID:
Flags: 0
{"kind": "exporter", "data_type": "logs", "name": "logging"}
Loki works fine
No traces exported for Tempo
