highIntermediateSymptom-first#rbac#serviceaccount#permissions#forbidden

RBAC Forbidden — serviceaccount cannot access API resources

kubectl logs show 'Error from server (Forbidden): pods is forbidden'

Likely source:securityconfiguration

Focused incident mode — strips away everything but this runbook.

Symptoms

  • kubectl logs show 'Error from server (Forbidden): pods is forbidden'
  • In-cluster application fails to list pods or secrets via the API
  • Controllers or operators report authorization errors in logs

Use kubectl auth can-i --list --as=system:serviceaccount:<ns>:<sa> to see ALL permissions for a service account. This is the fastest way to debug RBAC issues.

Evidence to grab

Collect these artifacts before you change anything — they prove (or rule out) each cause.

  • RBAC role bindings

    kubectl get rolebinding,clusterrolebinding -A | grep <serviceaccount>

  • ServiceAccount details

    kubectl get sa <sa> -n <namespace> -o yaml

Diagnostic flow

Run the steps in order, marking each claim confirmed or refuted. The evidence you gather feeds the ranked causes below.

Guided decision tree is ready.

Steps with commands, expected output, and pass/fail branches.

Ranked root causes

Evidence checklist

  • kubectl auth can-i returns no
  • No binding found for the SA

Confirm before fixing

kubectl auth can-i get pods --as=system:serviceaccount:<namespace>:<sa>
READ

Test SA permission

Fix

kubectl create role pod-reader --verb=get,list,watch --resource=pods -n <namespace> && kubectl create rolebinding pod-reader-binding --role=pod-reader --serviceaccount=<namespace>:<sa> -n <namespace>
CHANGE

Create a RoleBinding for the service account

Expected: rolebinding.rbac.authorization.k8s.io/pod-reader-binding created

Resolution runbook

Apply from top to bottom after the cause is confirmed. Review every command before running it — risk tags are shown on each.

1
kubectl auth can-i get pods --as=system:serviceaccount:<namespace>:<sa>
READ

Test the service account's permission

2
kubectl get rolebinding,clusterrolebinding -A -o yaml | grep -A5 <sa>
READ

Find existing bindings

3
kubectl create role <role> --verb=get,list,watch --resource=pods -n <namespace>
CHANGE

Create a Role granting the needed permissions

4
kubectl create rolebinding <name> --role=<role> --serviceaccount=<namespace>:<sa> -n <namespace>
CHANGE

Bind the role to the service account

5
kubectl auth can-i get pods --as=system:serviceaccount:<namespace>:<sa>
READ

Verify the permission now works

Prevention

  • Follow least-privilege: grant only the verbs and resources the SA needs.
  • Use ClusterRole + ClusterRoleBinding only when cross-namespace access is required.
  • Audit RBAC bindings regularly with kubectl auth can-i --list.
  • Use tools like rakkess or kubectl-who-can to review effective permissions.

Commands marked READ CHANGE DESTRUCTIVE — never run DESTRUCTIVE commands against production data you cannot restore. Copy → verify → execute.