highAdvancedSymptom-first#cert-manager#dns-01#acme#txt

DNS-01 challenge keeps failing — TXT record never propagates

Certificate not issued after waiting minutes

Likely source:configurationconnectivity

Focused incident mode — strips away everything but this runbook.

Symptoms

  • Certificate not issued after waiting minutes
  • Challenge state stays pending or shows 'TXT record not propagated'
  • dig shows no _acme-challenge TXT record for the domain

Evidence to grab

Collect these artifacts before you change anything — they prove (or rule out) each cause.

  • Challenge status

    kubectl get challenge -n <NAMESPACE> -o yaml | grep -A5 status
  • DNS provider credentials

    kubectl get secret <DNSPROVIDER-CREDS> -n cert-manager -o jsonpath='{.data}' | head -c 200

Diagnostic flow

Run the steps in order, marking each claim confirmed or refuted. The evidence you gather feeds the ranked causes below.

Guided decision tree is ready.

Steps with commands, expected output, and pass/fail branches.

Ranked root causes

Evidence checklist

  • Solver logs an authorization error for the zone
  • Secret token scoped to the wrong hosted zone/legacy API key

Confirm before fixing

kubectl get secret <DNSPROVIDER-CREDS> -n cert-manager -o jsonpath='{.data}' | wc -c
READ

Sanity-check credentials exist

Fix

kubectl -n cert-manager create secret generic <DNSPROVIDER-CREDS> --from-literal=access-key=<AK> --from-literal=secret-key=<SK> --dry-run=client -o yaml | kubectl apply -f -
CHANGE

Re-issue credentials with zone-scoped access

Resolution runbook

Apply from top to bottom after the cause is confirmed. Review every command before running it — risk tags are shown on each.

1
dig +short _acme-challenge.<DOMAIN>. TXT
READ

Is the record visible at the authoritative server?

2
kubectl get challenge -n <NAMESPACE> -o yaml | grep -B2 -A10 status
READ

Challenge state

3
kubectl logs -n cert-manager deploy/cert-manager --tail=200 | grep -i dns
READ

Solver errors (auth vs propagation)

Prevention

  • Use workload-identity (IRSA/Workload Identity) instead of static DNS secrets.
  • Set DNS zone TTL to 60s or lower for _acme-challenge domains.
  • Test issuance with a staging Issuer before production and before DNS changes.

Commands marked READ CHANGE DESTRUCTIVE — never run DESTRUCTIVE commands against production data you cannot restore. Copy → verify → execute.