highAdvancedSymptom-first#ebs#kms#encryption#attach

Encrypted EBS volume fails to attach — KMS permission problem

AttachVolume fails or volume stays in 'attaching' for encrypted volumes

Likely source:configurationsecurity

Focused incident mode — strips away everything but this runbook.

Symptoms

  • AttachVolume fails or volume stays in 'attaching' for encrypted volumes
  • CloudTrail shows kms:Decrypt AccessDenied for the volume's key
  • Launch of an instance with encrypted root volume times out or errors

Evidence to grab

Collect these artifacts before you change anything — they prove (or rule out) each cause.

  • Volume encryption key

    aws ec2 describe-volumes --volume-ids <VOLUME-ID> --query 'Volumes[0].{Encrypted:Encrypted,KmsKeyId:KmsKeyId}'
  • KMS key policy

    aws kms get-key-policy --key-id <KEY-ID> --policy-name default

Diagnostic flow

Run the steps in order, marking each claim confirmed or refuted. The evidence you gather feeds the ranked causes below.

Guided decision tree is ready.

Steps with commands, expected output, and pass/fail branches.

Ranked root causes

Evidence checklist

  • AccessDenied in CloudTrail on kms:CreateGrant or kms:Decrypt
  • Key is a customer-managed CMK with a restrictive policy

Confirm before fixing

aws kms get-key-policy --key-id <KEY-ID> --policy-name default
READ

Read the key policy

Fix

aws kms put-key-policy --key-id <KEY-ID> --policy-name default --policy file://key-policy.json
CHANGE

Add the principal to the key policy (example: arn:aws:iam::<ACCOUNT>:role/<ROLE> with kms:Decrypt and kms:CreateGrant)

Expected: The updated policy is applied

Resolution runbook

Apply from top to bottom after the cause is confirmed. Review every command before running it — risk tags are shown on each.

1
aws ec2 describe-volumes --volume-ids <VOLUME-ID> --query 'Volumes[0].{Encrypted:Encrypted,KmsKeyId:KmsKeyId}'
READ

Identify the key

2
aws kms describe-key --key-id <KEY-ID> --query 'KeyMetadata.{State:KeyState,Enabled:Enabled}'
READ

Confirm the key is usable

3
aws kms get-key-policy --key-id <KEY-ID> --policy-name default
READ

Review the policy before editing

4
aws kms put-key-policy --key-id <KEY-ID> --policy-name default --policy file://key-policy.json
CHANGE

Grant the attaching principal kms:Decrypt + kms:CreateGrant

5
aws ec2 attach-volume --volume-id <VOLUME-ID> --instance-id <INSTANCE-ID> --device <DEVICE-NAME>
CHANGE

Retry the attachment

Expected: in-use

Prevention

  • Give one IAM shape to every principal that mounts encrypted volumes (kms:Decrypt, kms:CreateGrant, kms:DescribeKey).
  • Use AWS-managed keys or a central key with a permissive-but-audited policy.
  • Pre-flight kms:describe-key before every attach script.

Commands marked READ CHANGE DESTRUCTIVE — never run DESTRUCTIVE commands against production data you cannot restore. Copy → verify → execute.