mediumIntermediateSymptom-first#cert-manager#http-01#ingress#acme

HTTP-01 challenge returns 404 — solver domain unreachable

Certificate not issuing via HTTP-01

Likely source:configurationconnectivity

Focused incident mode — strips away everything but this runbook.

Symptoms

  • Certificate not issuing via HTTP-01
  • Challenge events show 'http-01 solver for domain returned 404'
  • ACME endpoint can't reach http://<DOMAIN>/.well-known/acme-challenge/...

HTTP-01 needs the ACME server to reach port 80 of the domain from the public internet — any TLS redirect or missing port-80 route breaks it.

Evidence to grab

Collect these artifacts before you change anything — they prove (or rule out) each cause.

  • Challenge solver events

    kubectl describe challenge -n <NAMESPACE> | tail -30

  • Solver pod

    kubectl get pods -n <NAMESPACE> -l acme.cert-manager.io/http01-solver=true

Diagnostic flow

Run the steps in order, marking each claim confirmed or refuted. The evidence you gather feeds the ranked causes below.

Guided decision tree is ready.

Steps with commands, expected output, and pass/fail branches.

Ranked root causes

Evidence checklist

  • Other paths on the same Ingress work, but the ACME path 404s
  • Ingress has rewrite/redirect annotations that mangle the path

Confirm before fixing

kubectl get ingress <INGRESS> -n <NAMESPACE> -o jsonpath='{.metadata.annotations}'
READ

Inspect rewrite/redirect annotations

Fix

kubectl annotate ingress <INGRESS> -n <NAMESPACE> nginx.ingress.kubernetes.io/rewrite-target= --overwrite
CHANGE

Remove the aggressive rewrite, or point the solver service directly via a dedicated Ingress

Resolution runbook

Apply from top to bottom after the cause is confirmed. Review every command before running it — risk tags are shown on each.

1
curl -v http://<DOMAIN>/.well-known/acme-challenge/
READ

Reproduce the public failure

2
kubectl get ingress -A | grep <DOMAIN>
READ

Find the routing rule

3
kubectl describe challenge -n <NAMESPACE> | tail -30
READ

Read the solver error

Prevention

  • Prefer DNS-01 for robustness (no ingress dependency).
  • Keep a dedicated ACME solver Ingress with a catch-all path.
  • Verify the challenge path with curl before requesting the certificate.

Related runbooks

Same stack, same failure family — worth a skim.

Certificate stuck in NotReady — issuance failedcritical

kubectl get certificate shows Ready=False

cert-managerIntermediate3 causes
Open runbook
DNS-01 challenge keeps failing — TXT record never propagateshigh

Certificate not issued after waiting minutes

cert-managerAdvanced2 causes
Open runbook

Commands marked READ CHANGE DESTRUCTIVE — never run DESTRUCTIVE commands against production data you cannot restore. Copy → verify → execute.