criticalAdvancedSymptom-first#eks#aws-auth#rbac#authentication

EKS nodes/roles can't authenticate — aws-auth missing entry

kubectl returns 'could not get token: AccessDeniedException' for a role

Likely source:securityconfiguration

Focused incident mode — strips away everything but this runbook.

Symptoms

  • kubectl returns 'could not get token: AccessDeniedException' for a role
  • New nodegroup joins but workloads on it stay Pending (nodes themselves unauthenticated)
  • kubectl works for some roles but not the newly added one

EKS couples IAM identity to Kubernetes identity exclusively through aws-auth (plus SSO access entries in newer clusters). A new nodegroup is the #1 trigger for a missing mapRoles entry — treat it as part of the nodegroup rollout checklist.

Evidence to grab

Collect these artifacts before you change anything — they prove (or rule out) each cause.

  • aws-auth configmap

    kubectl get configmap -n kube-system aws-auth -o yaml

  • Cluster identity

    aws eks describe-cluster --name <CLUSTER-NAME> --query 'cluster.{Arn:arn,Identity:identity}'

Diagnostic flow

Run the steps in order, marking each claim confirmed or refuted. The evidence you gather feeds the ranked causes below.

Guided decision tree is ready.

Steps with commands, expected output, and pass/fail branches.

Ranked root causes

Evidence checklist

  • Nodes NeverReady after joining, /var/log/kubelet auth errors
  • aws-auth has mapRoles without the node rolearn

Confirm before fixing

kubectl get configmap -n kube-system aws-auth -o jsonpath='{.data.mapRoles}'
READ

Search for the node role ARN

Fix

kubectl edit configmap -n kube-system aws-auth
CHANGE

Add {"groups":["system:bootstrappers","system:nodes"],"rolearn":"<NODE-ROLE-ARN>","username":"system:node:{{EC2PrivateDNSName}}"} to mapRoles

Resolution runbook

Apply from top to bottom after the cause is confirmed. Review every command before running it — risk tags are shown on each.

1
kubectl get configmap -n kube-system aws-auth -o yaml
READ

Inspect the current mapping

2
aws sts get-caller-identity
READ

Match the exact ARN format

3
kubectl edit configmap -n kube-system aws-auth
CHANGE

Add the missing rolearn to mapRoles with the system:nodes group for node roles

4
kubectl get nodes
READ

Confirm nodes join / roles authenticate after a few minutes

Expected: Ready nodes or a resolved kubectl token flow

Prevention

  • Manage aws-auth in IaC (eksctl managed nodegroups write it for you).
  • Verify token+binding as a post-nodegroup smoke test (kubectl get nodes).
  • Switch accounts/roles deliberately; audit mapRoles on every role add.

Related runbooks

Same stack, same failure family — worth a skim.

EKS nodegroup instances not joining the clusterhigh

New nodegroup instances launch but never appear in kubectl get nodes

AWSProduction3 causes
Open runbook
New IAM role says 'not authorized' — propagation delaymedium

Freshly created role fails with AccessDenied/not authorized

AWSFoundation2 causes
Open runbook

Commands marked READ CHANGE DESTRUCTIVE — never run DESTRUCTIVE commands against production data you cannot restore. Copy → verify → execute.