criticalProductionSymptom-first#autoscaling#launch#quota#ami

New ASG instances failing to launch — quotas and AMI

ASG scale-out spins up instances that immediately terminate

Likely source:deploymentscaling

Focused incident mode — strips away everything but this runbook.

Symptoms

  • ASG scale-out spins up instances that immediately terminate
  • Scaling activities show Failed/NotStarted
  • Auto-recovery causes launch churn (instance count flaps)

Launch failures hide behind a healthy ASG — the group state looks fine while it silently sits at min size. Read the Failed activities before touching anything else.

Evidence to grab

Collect these artifacts before you change anything — they prove (or rule out) each cause.

  • Failure reason in scaling activities

    aws autoscaling describe-scaling-activities --auto-scaling-group-name <ASG-NAME> --max-items 10 --query 'Activities[?StatusCode==`Failed`].{Time:StartTime,Reason:Description}'

  • Launch template data

    aws ec2 describe-launch-template-versions --launch-template-id <LT-ID> --versions '$Latest'

Diagnostic flow

Run the steps in order, marking each claim confirmed or refuted. The evidence you gather feeds the ranked causes below.

Guided decision tree is ready.

Steps with commands, expected output, and pass/fail branches.

Ranked root causes

Evidence checklist

  • Failure message VolumeLimitExceeded or Max number of instances reached
  • Quota value at/under current fleet + ASG max

Confirm before fixing

aws service-quotas get-service-quota --service-code ec2 --quota-code L-1216C47A --region <REGION>
READ

Check the on-demand instance quota

Fix

aws service-quotas request-service-quota-increase --service-code ec2 --quota-code L-1216C47A --desired-value <NEW-QUOTA>
CHANGE

Request a raise (also raise EBS volume quota if VolumeLimitExceeded)

Expected: A case/reference number

Resolution runbook

Apply from top to bottom after the cause is confirmed. Review every command before running it — risk tags are shown on each.

1
aws autoscaling describe-scaling-activities --auto-scaling-group-name <ASG-NAME> --max-items 10 --query 'Activities[?StatusCode==`Failed`].Description'
READ

Pin the failure message

2
aws service-quotas get-service-quota --service-code ec2 --quota-code L-1216C47A --region <REGION>
READ

Quota headroom check

3
aws ec2 describe-images --image-ids <AMI-ID>
READ

AMI validity check

4
aws autoscaling update-auto-scaling-group --auto-scaling-group-name <ASG-NAME> --min-size <MIN> --max-size <MAX>
CHANGE

Once fixed, restore the capacity delta and let the ASG converge

Prevention

  • Automate quota requests to keep headroom above MaxSize.
  • Build AMIs region-resident and version them (never gold-master into one region only).
  • Add a post-scaling check that new instances stay Running 5+ minutes.

Related runbooks

Same stack, same failure family — worth a skim.

Auto Scaling group not scaling out under loadhigh

Load climbs but instance count stays flat

AWSIntermediate4 causes
Open runbook
InsufficientInstanceCapacity — unable to launch or scalehigh

ec2 run-instances returns InsufficientInstanceCapacity

AWSProduction2 causes
Open runbook
Subnet exhausted — no more IP addresses to launchhigh

run-instances fails: There aren't enough IP addresses available in subnet

AWSProductionUSE3 causes
Open runbook

Commands marked READ CHANGE DESTRUCTIVE — never run DESTRUCTIVE commands against production data you cannot restore. Copy → verify → execute.