Encrypted EBS volume fails to attach — KMS permission problem
AttachVolume fails or volume stays in 'attaching' for encrypted volumes
Focused incident mode — strips away everything but this runbook.
Symptoms
- AttachVolume fails or volume stays in 'attaching' for encrypted volumes
- CloudTrail shows kms:Decrypt AccessDenied for the volume's key
- Launch of an instance with encrypted root volume times out or errors
Evidence to grab
Collect these artifacts before you change anything — they prove (or rule out) each cause.
Volume encryption key
aws ec2 describe-volumes --volume-ids <VOLUME-ID> --query 'Volumes[0].{Encrypted:Encrypted,KmsKeyId:KmsKeyId}'KMS key policy
aws kms get-key-policy --key-id <KEY-ID> --policy-name default
Diagnostic flow
Run the steps in order, marking each claim confirmed or refuted. The evidence you gather feeds the ranked causes below.
Guided decision tree is ready.
Steps with commands, expected output, and pass/fail branches.
Ranked root causes
Evidence checklist
- AccessDenied in CloudTrail on kms:CreateGrant or kms:Decrypt
- Key is a customer-managed CMK with a restrictive policy
Confirm before fixing
aws kms get-key-policy --key-id <KEY-ID> --policy-name defaultRead the key policy
Fix
aws kms put-key-policy --key-id <KEY-ID> --policy-name default --policy file://key-policy.jsonAdd the principal to the key policy (example: arn:aws:iam::<ACCOUNT>:role/<ROLE> with kms:Decrypt and kms:CreateGrant)
Expected: The updated policy is applied
Resolution runbook
Apply from top to bottom after the cause is confirmed. Review every command before running it — risk tags are shown on each.
aws ec2 describe-volumes --volume-ids <VOLUME-ID> --query 'Volumes[0].{Encrypted:Encrypted,KmsKeyId:KmsKeyId}'Identify the key
aws kms describe-key --key-id <KEY-ID> --query 'KeyMetadata.{State:KeyState,Enabled:Enabled}'Confirm the key is usable
aws kms get-key-policy --key-id <KEY-ID> --policy-name defaultReview the policy before editing
aws kms put-key-policy --key-id <KEY-ID> --policy-name default --policy file://key-policy.jsonGrant the attaching principal kms:Decrypt + kms:CreateGrant
aws ec2 attach-volume --volume-id <VOLUME-ID> --instance-id <INSTANCE-ID> --device <DEVICE-NAME>Retry the attachment
Expected: in-use
Prevention
- Give one IAM shape to every principal that mounts encrypted volumes (kms:Decrypt, kms:CreateGrant, kms:DescribeKey).
- Use AWS-managed keys or a central key with a permissive-but-audited policy.
- Pre-flight kms:describe-key before every attach script.
Related runbooks
Same stack, same failure family — worth a skim.
Commands marked READ CHANGE DESTRUCTIVE — never run DESTRUCTIVE commands against production data you cannot restore. Copy → verify → execute.