criticalFoundationSymptom-first#pods#crashloopbackoff#restarts#startup

CrashLoopBackOff — container keeps crashing and restarting

kubectl get pods shows CrashLoopBackOff status

Likely source:availabilityconfiguration

Focused incident mode — strips away everything but this runbook.

Symptoms

  • kubectl get pods shows CrashLoopBackOff status
  • Pod restart count climbs with each cycle (10s, 20s, 40s...)
  • Container never reaches Ready/Running

Follow the exit code first: 137 → memory path, 1/x → application path, 0 → probe path. Restart backoff is exponential; do not blast-restart a crashing pod.

Evidence to grab

Collect these artifacts before you change anything — they prove (or rule out) each cause.

  • Pod status and restart count

    kubectl get pod <pod> -n <namespace> -o wide

  • Last terminated state

    kubectl get pod <pod> -n <namespace> -o jsonpath='{.status.containerStatuses[0].lastState}'

Diagnostic flow

Run the steps in order, marking each claim confirmed or refuted. The evidence you gather feeds the ranked causes below.

Guided decision tree is ready.

Steps with commands, expected output, and pass/fail branches.

Ranked root causes

Evidence checklist

  • kubectl logs <pod> --previous shows a stack trace before death
  • Exit code is 1 or another nonzero app code
  • Restarts follow a predictable back-off pattern

Confirm before fixing

kubectl logs <pod> -n <namespace> --previous | tail -50
READ

Tail previous logs for the crash reason

Expected: A panic or error message shortly before exit

Fix

kubectl set image deployment/<deploy> <container>=<image>:<fixed-tag>
CHANGE

Deploy a fixed image once the bug is patched

Expected: deployment.apps/<deploy> image updated

Resolution runbook

Apply from top to bottom after the cause is confirmed. Review every command before running it — risk tags are shown on each.

1
kubectl logs <pod> -n <namespace> --previous
READ

Get the logs from the crashed instance

2
kubectl get pod <pod> -n <namespace> -o jsonpath='{.status.containerStatuses[0].lastState.terminated.exitCode}'
READ

Check the exit code (137 = OOM, 1 = app error)

3
kubectl describe pod <pod> -n <namespace> | tail -30
READ

Review events for probes, mounts, and image pulls

4
kubectl top pod <pod> -n <namespace> --containers
READ

Check live memory/CPU usage to confirm OOM

Prevention

  • Set memory requests and limits with buffer above peak observed usage.
  • Set readiness + liveness probes with initialDelaySeconds larger than worst-case startup.
  • Use start meshing or startupProbe for slow-to-start applications.
  • Keep secrets and configmaps versioned in the same repo as the manifest.

Related runbooks

Same stack, same failure family — worth a skim.

OOMKilled containers — memory limit exceededcritical

Exit code 137 on the container

KubernetesFoundationUSE3 causes
Open runbook
ImagePullBackOff / ErrImagePull — pod can't fetch its imagehigh

Pod stuck in ImagePullBackOff or ErrImagePull

KubernetesFoundation3 causes
Open runbook
Pods stuck in Pending state — unschedulablehigh

kubectl get pods shows Pending for minutes

KubernetesIntermediate3 causes
Open runbook

Commands marked READ CHANGE DESTRUCTIVE — never run DESTRUCTIVE commands against production data you cannot restore. Copy → verify → execute.