AzureSenior

AKS pulls: pull QPS exceeded, 429 rate limits and stale tags

Kubelet's pull rate limit, Docker Hub's 429, and a reused tag can each break pulls. Use IfNotPresent, import images into ACR, and tag images immutably.

What you see

  • Failed to pull image "ACR/repo/nginx:latest": pull QPS exceeded.
  • 429 Too Many Requests or You have reached your pull rate limit from a public registry
  • manifest not found right after a tag was updated

What it means

Three different problems share the same symptom. The kubelet has its own default pull rate limit, so many jobs pulling the same image at once can exceed it. Public registries limit pulls per outbound IP, and a cluster shares one. And a mutable tag that is overwritten can leave nodes with a stale digest.

How to confirm the cause

  1. 1
    Read the exact message
    kubectl describe pod POD_NAME -n NAMESPACE

    pull QPS exceeded, 429, and manifest not found point to three different causes.

  2. 2
    Check the image pull policy
    kubectl get deployment NAME -o jsonpath='{.spec.template.spec.containers[*].imagePullPolicy}'

    Always on many pods makes every start a pull.

Causes and fixes

  • Kubelet pull rate limit exceeded

    HOW TO TELL
    pull QPS exceeded.
    ROOT CAUSE
    Many pods start at once and each pulls, exceeding the kubelet's registryPullQPS default.
    FIX
    Set imagePullPolicy: IfNotPresent, so images already on the node are not pulled again.
  • A public registry rate-limits the cluster

    HOW TO TELL
    429 Too Many Requests from Docker Hub or ghcr.io.
    ROOT CAUSE
    All nodes share one outbound IP address, so the cluster's pulls count together against the limit.
    FIX
    Import images into your own ACR, use ACR Artifact Cache as a pull-through cache, or authenticate with an image pull secret.
    az acr import --name REGISTRY --source docker.io/library/nginx:latest --image nginx:latest
  • Tag reuse leaves a stale digest

    HOW TO TELL
    manifest not found, or a pod runs old code after a push.
    ROOT CAUSE
    A mutable tag such as latest was overwritten. Nodes that cached the old digest may fail, or IfNotPresent may keep the old image.
    FIX
    Use unique, immutable tags such as the Git commit SHA. If you must reuse a tag, set imagePullPolicy: Always, and trigger a rolling restart to re-resolve it.
    kubectl rollout restart deployment/NAME -n NAMESPACE

How to stop it coming back

  • Tag every image with the commit SHA.
  • Mirror public base images into ACR.
  • Use IfNotPresent for immutable tags.

Keep going

Sources

Microsoft Learn (learn.microsoft.com), read 18 Nov 2026. The page's own metadata shows it was last updated 30 Oct 2026.

Search

Search cheat sheets and projects