criticalIntermediateSymptom-first#opentelemetry#traces#collector#otlp#connectivity

OpenTelemetry traces not arriving at the collector — export fails or batch processor aborts

Application exports traces but nothing appears in Jaeger/Tempo

Likely source:connectivityconfiguration

Focused incident mode — strips away everything but this runbook.

Symptoms

  • Application exports traces but nothing appears in Jaeger/Tempo
  • Collector logs show no incoming spans from the application
  • Application logs show 'OTLP export failed' or 'connection refused' to collector
  • Traces arrive for some services but not others

OTLP is the standard protocol for OpenTelemetry — always verify the endpoint, port (4318 for HTTP, 4317 for gRPC), and TLS settings first. The most common issue is a wrong URL or insecure flag mismatch.

Evidence to grab

Collect these artifacts before you change anything — they prove (or rule out) each cause.

  • Collector received spans metric

    curl -s http://<OTEL-COLLECTOR>:8888/metrics | grep 'otelcol_receiver_accepted_spans'

  • Application OTLP endpoint config

    kubectl get deploy -n <APP-NAMESPACE> <APP> -o yaml | grep -A3 OTEL_EXPORTER_OTLP_ENDPOINT

Diagnostic flow

Run the steps in order, marking each claim confirmed or refuted. The evidence you gather feeds the ranked causes below.

Guided decision tree is ready.

Steps with commands, expected output, and pass/fail branches.

Ranked root causes

Evidence checklist

  • Application OTEL_EXPORTER_OTLP_ENDPOINT points to the wrong host or port
  • Connection refused errors in application logs
  • The collector service port changed but the app config was not updated

Confirm before fixing

kubectl get deploy -n <APP-NAMESPACE> <APP> -o yaml | grep -A3 OTEL_EXPORTER_OTLP_ENDPOINT
READ

Read the OTLP endpoint from the deployment

Expected: A URL like http://otel-collector.observability:4318

Fix

kubectl set env deployment/<APP> -n <APP-NAMESPACE> OTEL_EXPORTER_OTLP_ENDPOINT=http://<OTEL-COLLECTOR>.<NAMESPACE>:4318
CHANGE

Update the OTLP endpoint URL

Expected: deployment.apps/<APP> env updated

Resolution runbook

Apply from top to bottom after the cause is confirmed. Review every command before running it — risk tags are shown on each.

1
curl -s http://<OTEL-COLLECTOR>:8888/metrics | grep 'otelcol_receiver_accepted_spans'
READ

Check if the collector is receiving spans

2
kubectl exec -n <APP-NAMESPACE> deploy/<APP> -- env | grep OTEL_EXPORTER_OTLP
READ

Check the OTLP endpoint and TLS settings in the application

3
kubectl exec -n <APP-NAMESPACE> deploy/<APP> -- curl -s -o /dev/null -w '%{http_code}' http://<OTEL-COLLECTOR>:4318/v1/traces
READ

Test connectivity from the app to the collector

4
kubectl set env deployment/<APP> -n <APP-NAMESPACE> OTEL_EXPORTER_OTLP_ENDPOINT=http://<OTEL-COLLECTOR>.<NAMESPACE>:4318
CHANGE

Correct the OTLP endpoint URL

Prevention

  • Use a Kubernetes Service DNS name for the collector endpoint (not IP addresses).
  • Add a startup probe to the application that verifies OTLP connectivity.
  • Monitor otelcol_receiver_accepted_spans and alert if it drops to 0 for > 5 minutes.
  • Use environment variable templating (e.g., Helm values) to keep OTLP config consistent across services.

Related runbooks

Same stack, same failure family — worth a skim.

OpenTelemetry traces lost — collector overload or context canceled on exporthigh

Traces arrive at the collector but do not appear in the backend (Jaeger/Tempo)

OpenTelemetryAdvancedUSE3 causes
Open runbook
OpenTelemetry collector fails to start — invalid YAML config or missing receivercritical

Collector pod shows CrashLoopBackOff after config change

OpenTelemetryIntermediate3 causes
Open runbook

Commands marked READ CHANGE DESTRUCTIVE — never run DESTRUCTIVE commands against production data you cannot restore. Copy → verify → execute.