AzureSenior
AKS pulls: pull QPS exceeded, 429 rate limits and stale tags
Kubelet's pull rate limit, Docker Hub's 429, and a reused tag can each break pulls. Use IfNotPresent, import images into ACR, and tag images immutably.
What you see
Failed to pull image "ACR/repo/nginx:latest": pull QPS exceeded.429 Too Many RequestsorYou have reached your pull rate limitfrom a public registrymanifest not foundright after a tag was updated
What it means
Three different problems share the same symptom. The kubelet has its own default pull rate limit, so many jobs pulling the same image at once can exceed it. Public registries limit pulls per outbound IP, and a cluster shares one. And a mutable tag that is overwritten can leave nodes with a stale digest.
How to confirm the cause
- 1Read the exact message
kubectl describe pod POD_NAME -n NAMESPACE
pull QPS exceeded,429, andmanifest not foundpoint to three different causes. - 2Check the image pull policy
kubectl get deployment NAME -o jsonpath='{.spec.template.spec.containers[*].imagePullPolicy}'Alwayson many pods makes every start a pull.
Causes and fixes
Kubelet pull rate limit exceeded
- HOW TO TELL
pull QPS exceeded.- ROOT CAUSE
- Many pods start at once and each pulls, exceeding the kubelet's
registryPullQPSdefault. - FIX
- Set
imagePullPolicy: IfNotPresent, so images already on the node are not pulled again.
A public registry rate-limits the cluster
- HOW TO TELL
429 Too Many Requestsfrom Docker Hub orghcr.io.- ROOT CAUSE
- All nodes share one outbound IP address, so the cluster's pulls count together against the limit.
- FIX
- Import images into your own ACR, use ACR Artifact Cache as a pull-through cache, or authenticate with an image pull secret.
az acr import --name REGISTRY --source docker.io/library/nginx:latest --image nginx:latest
Tag reuse leaves a stale digest
- HOW TO TELL
manifest not found, or a pod runs old code after a push.- ROOT CAUSE
- A mutable tag such as
latestwas overwritten. Nodes that cached the old digest may fail, orIfNotPresentmay keep the old image. - FIX
- Use unique, immutable tags such as the Git commit SHA. If you must reuse a tag, set
imagePullPolicy: Always, and trigger a rolling restart to re-resolve it.kubectl rollout restart deployment/NAME -n NAMESPACE
How to stop it coming back
- Tag every image with the commit SHA.
- Mirror public base images into ACR.
- Use
IfNotPresentfor immutable tags.
Keep going
- Runbook: AKS ImagePullBackOff from ACR: 401 Unauthorized
- Runbook: You have reached your pull rate limit (Docker Hub 429)
- Cheat sheet: Azure CLI
- Cheat sheet: kubectl
- Lesson: the containers topic in Learn
Sources
Microsoft Learn (learn.microsoft.com), read 18 Nov 2026. The page's own metadata shows it was last updated 30 Oct 2026.