highAdvancedSymptom-first#persistentvolume#storage#reclaim#released

PersistentVolume stuck in Released state — reclaim not happening

kubectl get pv shows status Released for extended period

Likely source:availabilitydata-loss

Focused incident mode — strips away everything but this runbook.

Symptoms

  • kubectl get pv shows status Released for extended period
  • New PVC cannot bind to the PV even though it matches the spec
  • PersistentVolumeReclaimPolicy set to Delete but volume is not cleaned up

Released PVs with Delete policy should be cleaned up by the volume controller. If they are not, the controller is usually crashed or the CSI driver needs attention.

Evidence to grab

Collect these artifacts before you change anything — they prove (or rule out) each cause.

  • PV status and reclaim policy

    kubectl get pv <pv> -o jsonpath='{.status.phase},{.spec.persistentVolumeReclaimPolicy}'

  • PV claimRef

    kubectl get pv <pv> -o jsonpath='{.spec.claimRef}'

Diagnostic flow

Run the steps in order, marking each claim confirmed or refuted. The evidence you gather feeds the ranked causes below.

Guided decision tree is ready.

Steps with commands, expected output, and pass/fail branches.

Ranked root causes

Evidence checklist

  • PV is Released and claimRef still points to the deleted PVC
  • New PVC with matching spec does not bind

Confirm before fixing

kubectl get pv <pv> -o jsonpath='{.spec.claimRef}'
READ

Read the claimRef

Fix

kubectl patch pv <pv> -p '{"spec":{"claimRef":null}}'
CHANGE

Clear the claimRef so the PV becomes Available

Expected: persistentvolume/<pv> patched

Resolution runbook

Apply from top to bottom after the cause is confirmed. Review every command before running it — risk tags are shown on each.

1
kubectl get pv <pv> -o jsonpath='status={.status.phase} policy={.spec.persistentVolumeReclaimPolicy}'
READ

Read the status and reclaim policy

2
kubectl get pv <pv> -o jsonpath='{.spec.claimRef}'
READ

Check for stale claimRef

3
kubectl patch pv <pv> -p '{"spec":{"claimRef":null}}'
CHANGE

Clear the claimRef to make the PV Available

4
kubectl get pv <pv> -o jsonpath='{.status.phase}'
READ

Confirm the PV transitioned to Available

Prevention

  • Monitor PV phase and alert on Released state longer than 5 minutes.
  • Use Delete reclaim policy unless the volume contains data that must survive PVC deletion.
  • Ensure CSI controller pods have PDBs and are monitored.
  • Test PVC create/delete cycles in staging to validate reclaim works.

Related runbooks

Same stack, same failure family — worth a skim.

PersistentVolumeClaim stuck in Pending — no provisionerhigh

kubectl get pvc shows Pending

KubernetesAdvanced2 causes
Open runbook
PV reclaim slow or stuck — controller not picking up Released volumeshigh

PV stays in Released state for hours after PVC deletion

KubernetesAdvanced3 causes
Open runbook
Namespace stuck in Terminating — finalizers blocking deletionmedium

kubectl get namespace shows Terminating for minutes or hours

KubernetesAdvanced3 causes
Open runbook

Commands marked READ CHANGE DESTRUCTIVE — never run DESTRUCTIVE commands against production data you cannot restore. Copy → verify → execute.