mediumIntermediateUSE#zombie#orphan#wait#process#reap

Zombie processes accumulating — orphaned children not reaped

ps shows processes with Z state (zombie)

Likely source:performanceavailability

Focused incident mode — strips away everything but this runbook.

Symptoms

  • ps shows processes with Z state (zombie)
  • System process count keeps growing slowly
  • Application spawns children that exit but never disappear from ps

A few zombies are normal and harmless — they consume no CPU or memory, only a PID entry. Hundreds or thousands of zombies, however, indicate a real bug in the parent process.

Evidence to grab

Collect these artifacts before you change anything — they prove (or rule out) each cause.

  • Zombie process count

    ps -eo state | grep -c '^Z'

  • Zombie details

    ps -ef | grep 'Z' | head -10

Diagnostic flow

Run the steps in order, marking each claim confirmed or refuted. The evidence you gather feeds the ranked causes below.

Guided decision tree is ready.

Steps with commands, expected output, and pass/fail branches.

Ranked root causes

Evidence checklist

  • Many zombies share the same PPID
  • The parent is a long-running daemon or application server

Confirm before fixing

ps -eo ppid,stat | grep Z | awk '{print $1}' | sort | uniq -c | sort -rn | head -3
READ

Find which parent PID is leaking zombies

Fix

kill -TERM <parent-PID>
CHANGE

Restart the parent so it reaps zombies on exit

Resolution runbook

Apply from top to bottom after the cause is confirmed. Review every command before running it — risk tags are shown on each.

1
ps -eo pid,ppid,stat,cmd | grep '^\s*[0-9].*Z'
READ

List all zombie processes

2
ps -eo ppid,stat | grep Z | awk '{print $1}' | sort | uniq -c | sort -rn | head -3
READ

Find the parent PID leaking zombies

3
kill -TERM <parent-PID>
CHANGE

Restart the parent to force zombie reaping

Prevention

  • Fix application code to properly call wait() or waitpid() on child processes.
  • Use prctl(PR_SET_CHILD_SUBREAPER) in parent processes to adopt orphaned children.
  • Alert on zombie count exceeding 50 via a cron job or monitoring agent.
  • Restart services periodically to flush accumulated zombies.

Related runbooks

Same stack, same failure family — worth a skim.

Fork bomb — runaway process count exhausting PIDscritical

Cannot fork or Cannot allocate memory when trying to start new processes

LinuxIntermediateUSE2 causes
Open runbook
Process ignores SIGTERM — cannot be gracefully stoppedmedium

systemctl stop <service> hangs for the full timeout

LinuxIntermediate3 causes
Open runbook

Commands marked READ CHANGE DESTRUCTIVE — never run DESTRUCTIVE commands against production data you cannot restore. Copy → verify → execute.