criticalIntermediateUSE#aks#nodepool#notready#kubelet#health

AKS node pool NotReady — nodes failing health checks

kubectl get nodes shows NotReady for nodes in a specific pool

Likely source:availabilityperformance

Focused incident mode — strips away everything but this runbook.

Symptoms

  • kubectl get nodes shows NotReady for nodes in a specific pool
  • Pods in the affected pool are not scheduled or are evicted
  • az aks nodepool show shows node status as NotReady

AKS managed nodes have a kubelet auto-repair — if NotReady persists beyond 15 minutes, the platform will attempt a self-heal. If it does not, open a support ticket.

Evidence to grab

Collect these artifacts before you change anything — they prove (or rule out) each cause.

  • Node conditions

    kubectl describe node <NODE> | grep -A15 Conditions

  • AKS node pool status

    az aks nodepool show --resource-group <RG> --cluster-name <AKS> --name <POOL> --query "{state:powerState}" -o json

Diagnostic flow

Run the steps in order, marking each claim confirmed or refuted. The evidence you gather feeds the ranked causes below.

Guided decision tree is ready.

Steps with commands, expected output, and pass/fail branches.

Ranked root causes

Evidence checklist

  • Node condition Ready=False with message KubeletNotReady
  • az vmss run-command shows kubelet service failed

Confirm before fixing

az vmss run-command invoke --resource-group MC_<RG>_<AKS>_<LOCATION> --name <VMSS-NAME> --command-id RunShellScript --instance-id <INSTANCE-ID> --scripts "systemctl status kubelet" --query "output" -o tsv
READ

Check kubelet service status on the node

Fix

az vmss run-command invoke --resource-group MC_<RG>_<AKS>_<LOCATION> --name <VMSS-NAME> --command-id RunShellScript --instance-id <INSTANCE-ID> --scripts "sudo systemctl restart kubelet"
CHANGE

Restart kubelet on the affected node

Resolution runbook

Apply from top to bottom after the cause is confirmed. Review every command before running it — risk tags are shown on each.

1
kubectl get nodes -o wide | grep NotReady
READ

Identify affected nodes

2
kubectl describe node <NODE> | grep -A15 Conditions
READ

Read the failing condition

3
az vmss run-command invoke --resource-group MC_<RG>_<AKS>_<LOCATION> --name <VMSS-NAME> --command-id RunShellScript --instance-id <INSTANCE-ID> --scripts "sudo systemctl restart kubelet"
CHANGE

Restart kubelet if it stopped

4
kubectl get nodes
READ

Confirm nodes return to Ready

Prevention

  • Enable Azure Monitor for Containers to get kubelet health alerts.
  • Use multiple node pools for blast radius isolation.
  • Set up node auto-repair and monitor the kubelet auto-repair cycle.
  • Keep AKS version current to benefit from platform stability fixes.

Related runbooks

Same stack, same failure family — worth a skim.

AKS upgrade stuck — cluster not in Succeeded statehigh

az aks show returns provisioningState = Upgrading for more than 30 minutes

AzureAdvanced2 causes
Open runbook
Worker node NotReady — node down or kubelet failurecritical

kubectl get nodes shows NotReady for one or more workers

KubernetesAdvancedUSE3 causes
Open runbook

Commands marked READ CHANGE DESTRUCTIVE — never run DESTRUCTIVE commands against production data you cannot restore. Copy → verify → execute.