DNS-01 challenge keeps failing — TXT record never propagates
Certificate not issued after waiting minutes
Focused incident mode — strips away everything but this runbook.
Symptoms
- Certificate not issued after waiting minutes
- Challenge state stays pending or shows 'TXT record not propagated'
- dig shows no _acme-challenge TXT record for the domain
Evidence to grab
Collect these artifacts before you change anything — they prove (or rule out) each cause.
Challenge status
kubectl get challenge -n <NAMESPACE> -o yaml | grep -A5 statusDNS provider credentials
kubectl get secret <DNSPROVIDER-CREDS> -n cert-manager -o jsonpath='{.data}' | head -c 200
Diagnostic flow
Run the steps in order, marking each claim confirmed or refuted. The evidence you gather feeds the ranked causes below.
Guided decision tree is ready.
Steps with commands, expected output, and pass/fail branches.
Ranked root causes
Evidence checklist
- Solver logs an authorization error for the zone
- Secret token scoped to the wrong hosted zone/legacy API key
Confirm before fixing
kubectl get secret <DNSPROVIDER-CREDS> -n cert-manager -o jsonpath='{.data}' | wc -cSanity-check credentials exist
Fix
kubectl -n cert-manager create secret generic <DNSPROVIDER-CREDS> --from-literal=access-key=<AK> --from-literal=secret-key=<SK> --dry-run=client -o yaml | kubectl apply -f -Re-issue credentials with zone-scoped access
Resolution runbook
Apply from top to bottom after the cause is confirmed. Review every command before running it — risk tags are shown on each.
dig +short _acme-challenge.<DOMAIN>. TXTIs the record visible at the authoritative server?
kubectl get challenge -n <NAMESPACE> -o yaml | grep -B2 -A10 statusChallenge state
kubectl logs -n cert-manager deploy/cert-manager --tail=200 | grep -i dnsSolver errors (auth vs propagation)
Prevention
- Use workload-identity (IRSA/Workload Identity) instead of static DNS secrets.
- Set DNS zone TTL to 60s or lower for _acme-challenge domains.
- Test issuance with a staging Issuer before production and before DNS changes.
Related runbooks
Same stack, same failure family — worth a skim.
Commands marked READ CHANGE DESTRUCTIVE — never run DESTRUCTIVE commands against production data you cannot restore. Copy → verify → execute.