highIntermediateUSE#loki#logql#timeout#query#performance

Loki LogQL range query times out — context deadline exceeded

Loki query returns 'context deadline exceeded' or 'query timeout'

Likely source:performancescaling

Focused incident mode — strips away everything but this runbook.

Symptoms

  • Loki query returns 'context deadline exceeded' or 'query timeout'
  • Grafana Explore panel spins indefinitely when querying Loki
  • Loki querier pods show high CPU or memory usage during the query
  • The query works for a short time range but fails for longer ranges (e.g., > 24h)

Loki queries are bounded by max_query_length and query_timeout — not by the data volume. A 30-day query on a low-volume tenant may time out just as easily as a 1-day query on a high-volume tenant. Always test with a small limit first.

Evidence to grab

Collect these artifacts before you change anything — they prove (or rule out) each cause.

  • Loki query-frontend logs

    kubectl logs -n logging -l app=loki -c query-frontend | grep -i 'deadline exceeded\|timeout' | tail -10

  • Loki query-frontend config

    kubectl get cm -n logging loki-config -o yaml | grep -A5 'max_query_length\|query_timeout'

Diagnostic flow

Run the steps in order, marking each claim confirmed or refuted. The evidence you gather feeds the ranked causes below.

Guided decision tree is ready.

Steps with commands, expected output, and pass/fail branches.

Ranked root causes

Evidence checklist

  • The query covers > 7 days (or whatever max_query_length is set to)
  • error log shows 'time range exceeds max query length'
  • Shorter queries (e.g., 1h) work fine

Confirm before fixing

curl -s -G http://<LOKI-SERVER>:3100/loki/api/v1/query_range --data-urlencode 'query={job="<JOB>"}' --data-urlencode 'start=<START-7DAYS-AGO>' --data-urlencode 'end=<NOW>' --data-urlencode 'limit=1' 2>&1 | head -5
READ

Test a 7-day query

Expected: A response (may be empty but not a timeout if within limits)

Fix

kubectl edit cm -n logging loki-config
CHANGE

Increase max_query_length and query_timeout in the Loki config

Resolution runbook

Apply from top to bottom after the cause is confirmed. Review every command before running it — risk tags are shown on each.

1
kubectl logs -n logging -l app=loki -c query-frontend | grep -i 'deadline exceeded\|timeout' | tail -10
READ

Check for timeout errors in query-frontend logs

2
kubectl get cm -n logging loki-config -o yaml | grep -A5 -E 'query_timeout|max_query_length'
READ

Review timeout configuration

3
curl -s -G http://<LOKI-SERVER>:3100/loki/api/v1/query_range --data-urlencode 'query={job="<JOB>"}' --data-urlencode 'start=<START-EPOCH>' --data-urlencode 'end=<END-EPOCH>' --data-urlencode 'limit=10' | jq '.stats'
READ

Test a simplified query

4
kubectl edit cm -n logging loki-config
CHANGE

Increase query_timeout and max_query_length

Prevention

  • Set max_query_length to match your retention policy (e.g., 721h for 30-day retention).
  • Set query_timeout to at least 60s for dashboards, 300s for Explore.
  • Use query-frontend results caching to speed up repeated queries.
  • Alert on Loki query latency (loki_query_duration_seconds p99 > 30s).

Related runbooks

Same stack, same failure family — worth a skim.

Logs land in Loki but Grafana panel shows nothing — label matcher mismatchmedium

Grafana Loki panel shows 'No logs found' but the Loki API returns data

LokiIntermediate3 causes
Open runbook
Loki returns 429 — ingester rate limit or query entry limit exceededhigh

Loki returns HTTP 429 with 'maximum entries limit per query exceeded'

LokiIntermediateUSE3 causes
Open runbook

Commands marked READ CHANGE DESTRUCTIVE — never run DESTRUCTIVE commands against production data you cannot restore. Copy → verify → execute.