highIntermediateSymptom-first#runner#tags#queue#stuck

GitLab job stuck 'waiting for a runner' - tag mismatch

Job stuck in 'pending' state with 'Waiting for a runner'

Likely source:availabilityconfiguration

Focused incident mode — strips away everything but this runbook.

Symptoms

  • Job stuck in 'pending' state with 'Waiting for a runner'
  • GitLab UI shows 'This job is stuck because you don't have any runners with this tags'
  • Pipeline hangs indefinitely without any runner picking up the job

GitLab matches tags with AND logic by default. A job with tags: [docker, linux] requires a runner that has BOTH tags, not just one.

Evidence to grab

Collect these artifacts before you change anything — they prove (or rule out) each cause.

  • Job tags in .gitlab-ci.yml

    grep -A5 'tags:' .gitlab-ci.yml

  • Available runners and their tags

    glab api /runners --jq '.[] | {id, description, tags, active}'

Diagnostic flow

Run the steps in order, marking each claim confirmed or refuted. The evidence you gather feeds the ranked causes below.

Guided decision tree is ready.

Steps with commands, expected output, and pass/fail branches.

Ranked root causes

Evidence checklist

  • Job requires tags: [self-hosted, gpu] but no runner has gpu tag
  • glab api /runners shows no matching tags

Confirm before fixing

glab api /runners --jq '.[] | select(.tags | contains(["gpu"]))'
READ

Filter runners by the required tag

Fix

gitlab-runner register --url https://gitlab.com/ --token <RUNNER_TOKEN> --tag-list gpu,docker
CHANGE

Register a new runner with the required tag

Resolution runbook

Apply from top to bottom after the cause is confirmed. Review every command before running it — risk tags are shown on each.

1
glab api /runners --jq '.[] | {id, tags, active, online}'
READ

List all runners and their status

2
grep -B2 -A10 'tags:' .gitlab-ci.yml | head -40
READ

Read the job required tags

3
glab api /projects/<PROJECT_ID>/runners --jq '.[] | {id, tags}'
READ

Check runners available to this project

Prevention

  • Use a shared runner tag (e.g., docker) for generic jobs and only add specific tags when needed.
  • Add a CI/CD lint step that validates tag availability before merge.
  • Monitor runner availability via the GitLab runners API and alert on offline runners.
  • Document which runner tags are available and their executors in the project wiki.

Related runbooks

Same stack, same failure family — worth a skim.

GitLab CI image pull failed - private registry or missing imagehigh

Job fails with 'Error response from daemon: manifest for <image> not found'

GitLab CIIntermediate2 causes
Open runbook
GitLab pipeline blocked - manual job with allow_failure: falsemedium

Pipeline shows 'Blocked' status and cannot proceed

GitLab CIFoundation2 causes
Open runbook

Commands marked READ CHANGE DESTRUCTIVE — never run DESTRUCTIVE commands against production data you cannot restore. Copy → verify → execute.