criticalIntermediateRED#alb#elbv2#503#targets

ALB returns 503 — no healthy targets

HTTP 503 Service Unavailable from the load balancer

Likely source:availabilityconnectivity

Focused incident mode — strips away everything but this runbook.

Symptoms

  • HTTP 503 Service Unavailable from the load balancer
  • Target group shows 0/2 healthy in the console
  • All backends appear to run but get no traffic

ALB drains nothing when it has no healthy targets — it answers 503 immediately. Restore a healthy backend BEFORE rerouting DNS so you never serve bare 503 to users.

Evidence to grab

Collect these artifacts before you change anything — they prove (or rule out) each cause.

  • Target health state

    aws elbv2 describe-target-health --target-group-arn <TG-ARN> --query 'TargetHealthDescriptions[].{Target:Target.Id,State:TargetHealth.State,Reason:TargetHealth.Reason}'

  • LB state and listeners

    aws elbv2 describe-load-balancers --load-balancer-arns <LB-ARN> --query 'LoadBalancers[0].{State:State.Code,Type:Type}'

Diagnostic flow

Run the steps in order, marking each claim confirmed or refuted. The evidence you gather feeds the ranked causes below.

Guided decision tree is ready.

Steps with commands, expected output, and pass/fail branches.

Ranked root causes

Evidence checklist

  • Every target reports unhealthy with a consistent reason
  • Direct curl to a backend instance reproduces the failure

Confirm before fixing

aws ssm send-command --instance-ids <INSTANCE-ID> --document-name AWS-RunShellScript --parameters 'commands=["curl -sv http://localhost:<PORT><PATH>"]'
READ

Test the health path locally on a target

Expected: The failing status (4xx/5xx, redirect, or timeout)

A 4xx/redirect usually means the health path is wrong for the app; a 5xx/timeout means the app itself is down.

Fix

aws elbv2 modify-target-group --target-group-arn <TG-ARN> --health-check-path <CORRECT-PATH>
CHANGE

Correct the health check (path/port/protocol) to what the app actually serves

Expected: healthy

Resolution runbook

Apply from top to bottom after the cause is confirmed. Review every command before running it — risk tags are shown on each.

1
aws elbv2 describe-target-health --target-group-arn <TG-ARN>
READ

Enumerate target states and reasons

2
aws ssm send-command --instance-ids <INSTANCE-ID> --document-name AWS-RunShellScript --parameters 'commands=["curl -sv http://localhost:<PORT>/health"]'
READ

Reproduce the health check locally

3
aws elbv2 modify-target-group --target-group-arn <TG-ARN> --health-check-path /healthz
CHANGE

Fix path/port/protocol to match the app

4
watch -n 5 aws elbv2 describe-target-health --target-group-arn <TG-ARN> --query 'TargetHealthDescriptions[].TargetHealth.State'
READ

Watch targets flip healthy

Prevention

  • Make the health check a real app endpoint that also validates dependencies.
  • Keep at least 2 instances/AZ so a single unhealthy target never zeros the group.
  • Alert on healthy host count < 1 before the 503 reaches users.

Related runbooks

Same stack, same failure family — worth a skim.

ALB target unhealthy — health check configuration drifthigh

One target shows unhealthy while others stay healthy

AWSIntermediateRED3 causes
Open runbook
ALB returns 504 Gateway Timeout — backend too slowhigh

Requests return 504 from the load balancer at steady rate

AWSAdvancedRED3 causes
Open runbook

Commands marked READ CHANGE DESTRUCTIVE — never run DESTRUCTIVE commands against production data you cannot restore. Copy → verify → execute.