KubernetesSenior

kubectl drain hangs: a PodDisruptionBudget blocks eviction

kubectl drain respects PodDisruptionBudgets, so a strict budget can stall a node upgrade. Find the budget, then fix replicas or the budget, not the drain.

What you see

  • kubectl drain never finishes and keeps retrying evictions
  • A node upgrade or scale-down is stuck waiting on one pod

What it means

Drain evicts pods safely: it lets their containers terminate gracefully and respects the PodDisruptionBudgets you have set. If evicting a pod would drop its workload below the budget, the eviction is refused and drain waits.

A budget of minAvailable: 100%, or one equal to the replica count, can never be satisfied during a drain. That is a configuration problem, not a drain bug.

How to confirm the cause

  1. 1
    List budgets and how many disruptions they allow
    kubectl get pdb -A

    ALLOWED DISRUPTIONS of 0 is the blocker.

  2. 2
    See the budget's detail
    kubectl describe pdb PDB_NAME -n NAMESPACE

    Check the selector, the minimum or maximum, and the current healthy count.

  3. 3
    Check the workload has enough healthy replicas
    kubectl get deployment NAME -n NAMESPACE

    A single-replica workload with a budget cannot be drained.

Causes and fixes

  • The budget is too strict for the replica count

    HOW TO TELL
    ALLOWED DISRUPTIONS is 0 even though all pods are healthy.
    ROOT CAUSE
    The budget requires every pod to stay available, so no eviction is ever allowed.
    FIX
    Run more replicas than the budget's minimum, or relax the budget, so at least one pod can be evicted at a time.
  • Pods are already unhealthy

    HOW TO TELL
    Some pods are not Ready, so the healthy count is at the minimum.
    ROOT CAUSE
    Evicting a healthy pod would drop below the budget while another is already down.
    FIX
    Fix the unhealthy pods. The docs recommend setting the AlwaysAllow unhealthy pod eviction policy on budgets so misbehaving pods can be evicted during a drain.
  • DaemonSet pods block the drain

    HOW TO TELL
    Drain stops with a message about DaemonSet-managed pods.
    ROOT CAUSE
    DaemonSet pods are ignored by design and cannot be evicted normally.
    FIX
    Pass --ignore-daemonsets, as the docs show.
    kubectl drain --ignore-daemonsets NODE_NAME

How to stop it coming back

  • Set budgets as maxUnavailable: 1 for most workloads, not 100% available.
  • Run at least two replicas of anything with a budget.
  • Test a drain in staging before a maintenance window.

Keep going

Sources

Kubernetes documentation (kubernetes.io), read 18 Nov 2026. The docs version selector listed v1.33 to v1.37.

Search

Search cheat sheets and projects