Memory pressure — kswapd high, swap active, system thrashing
System is sluggish and unresponsive under load
Focused incident mode — strips away everything but this runbook.
Symptoms
- System is sluggish and unresponsive under load
- free -h shows swap is heavily used
- kswapd process is consuming significant CPU (top)
Swap thrashing can cascade into total system unresponsiveness. Always address the root memory consumer rather than just adding more swap.
Evidence to grab
Collect these artifacts before you change anything — they prove (or rule out) each cause.
Memory and swap usage
free -hSwap activity and page faults
vmstat 1 5
Diagnostic flow
Run the steps in order, marking each claim confirmed or refuted. The evidence you gather feeds the ranked causes below.
Guided decision tree is ready.
Steps with commands, expected output, and pass/fail branches.
Ranked root causes
Evidence checklist
- ps shows one or more processes with high RSS
- free -h shows available near 0
- SwapIn is non-zero in vmstat
Confirm before fixing
ps aux --sort=-%mem | head -5Identify the top memory consumers
Fix
kill -TERM <PID>Gracefully stop the runaway process
Resolution runbook
Apply from top to bottom after the cause is confirmed. Review every command before running it — risk tags are shown on each.
free -hAssess the memory deficit
vmstat 1 5Measure swap thrashing activity
ps aux --sort=-%mem | head -10Find the biggest memory consumers
kill -TERM <excessive-PID>Stop the offending process
Prevention
- Set appropriate memory limits (ulimit, cgroups) for each service.
- Alert on swap usage exceeding 25% via node_exporter.
- Tune vm.swappiness based on workload (10-30 for databases, 60 for general).
- Use memory profiling to find leaks before they cause thrashing.
Related runbooks
Same stack, same failure family — worth a skim.
Commands marked READ CHANGE DESTRUCTIVE — never run DESTRUCTIVE commands against production data you cannot restore. Copy → verify → execute.