highIntermediateSymptom-first#pods#scheduling#resources#pending

Pods stuck in Pending state — unschedulable

kubectl get pods shows Pending for minutes

Likely source:availabilityscaling

Focused incident mode — strips away everything but this runbook.

Symptoms

  • kubectl get pods shows Pending for minutes
  • Deployment never reaches desired replica count
  • describe shows FailedScheduling events

Evidence to grab

Collect these artifacts before you change anything — they prove (or rule out) each cause.

  • FailedScheduling reasons

    kubectl describe pod <pod> -n <namespace> | grep -A15 Events

  • Node capacity vs requests

    kubectl describe nodes | grep -A8 Allocated resources

Diagnostic flow

Run the steps in order, marking each claim confirmed or refuted. The evidence you gather feeds the ranked causes below.

Guided decision tree is ready.

Steps with commands, expected output, and pass/fail branches.

Ranked root causes

Evidence checklist

  • Event: 0/5 nodes are available: Insufficient cpu
  • Node capacity is almost fully allocated

Confirm before fixing

kubectl describe nodes | grep -A8 'Allocated resources'
READ

Review per-node allocation

Fix

kubectl scale deployment/<deploy> --replicas=<n>
CHANGE

Reduce replicas, or raise limits and let the autoscaler add nodes

Expected: deployment.apps/<deploy> scaled

Resolution runbook

Apply from top to bottom after the cause is confirmed. Review every command before running it — risk tags are shown on each.

1
kubectl describe pod <pod> -n <namespace> | grep -A15 Events
READ

Read the exact reason the scheduler refused

2
kubectl get nodes -o custom-columns=NAME:.metadata.name,TAINTS:.spec.taints
READ

Compare taints against the pod tolerations

3
kubectl get pvc -n <namespace>
READ

Confirm storage is Bound if volumes are involved

4
kubectl describe nodes | grep -A8 'Allocated resources'
READ

Confirm the cluster has room for the request

Prevention

  • Set explicit resource requests/limits and monitor node Allocatable.
  • Use cluster-autoscaler or node pools sized for surge.
  • Document taints/tolerations and node selectors as part of manifests.
  • Use PVCs with default storage class and monitor for Pending before deploy.

Related runbooks

Same stack, same failure family — worth a skim.

CrashLoopBackOff — container keeps crashing and restartingcritical

kubectl get pods shows CrashLoopBackOff status

KubernetesFoundation4 causes
Open runbook
PersistentVolumeClaim stuck in Pending — no provisionerhigh

kubectl get pvc shows Pending

KubernetesAdvanced2 causes
Open runbook
Worker node NotReady — node down or kubelet failurecritical

kubectl get nodes shows NotReady for one or more workers

KubernetesAdvancedUSE3 causes
Open runbook

Commands marked READ CHANGE DESTRUCTIVE — never run DESTRUCTIVE commands against production data you cannot restore. Copy → verify → execute.