Lesson 5 of 8 in CI/CD pipelines

Secrets and variables in pipelines

Core9 min

A secret is a value such as a token or password that the pipeline needs but that must never appear in code or logs. A variable is a plain configuration value that is fine to show. CI services store both outside the repository and inject them into a job when it runs.

Draft. This lesson is written and waiting for a person to run every command and review it. It is safe to learn from, but check the official documentation for anything you depend on.

Read these first: Writing a GitHub Actions workflow

The problem this solves

A deploy step needs credentials, and the easy mistake is to paste them into the workflow file or the repository. Anyone with read access to the code then has them, and git history keeps them even after you delete the line.

Storing a secret in the CI service solves the first half. The second half is the log: a careless echo or a failing command that prints its environment can copy the secret into a log that many people can read.

How it works

A secret reaches one step and is hidden in the log
A secret reaches one step and is hidden in the logThe secret is stored outside the repository, passed to a single step as an environment variable, and the runner replaces exact matches of it with asterisks in the log.1 jobrequests it2 pass asenv var3 output ismaskedSecret storerepo or environmentJobruns on the runnerStepenv: TOKENLogexact matches show ***
  • Security
  • Runs code
  • Data

The secret is stored outside the repository, passed to a single step as an environment variable, and the runner replaces exact matches of it with asterisks in the log.

  1. Secret store to Job: 1 job requests it
  2. Job to Step: 2 pass as env var
  3. Step to Log: 3 output is masked
OIDC replaces a stored cloud key
OIDC replaces a stored cloud keyThe job asks GitHub for a signed token, sends it to the cloud provider, and the cloud provider checks its trust rules and grants short-lived access.1 ask for token2 send token3 grant ifit matchesJobid-token: writeGitHub OIDCissues signed tokenCloud providerchecks trust rulesShort-lived accessexpires on its own
  • Runs code
  • Security
  • Other

The job asks GitHub for a signed token, sends it to the cloud provider, and the cloud provider checks its trust rules and grants short-lived access.

  1. Job to GitHub OIDC: 1 ask for token
  2. Job to Cloud provider: 2 send token
  3. Cloud provider to Short-lived access: 3 grant if it matches

Secrets can be defined for a repository, for an organization, or for one environment such as production. An environment secret is only given to jobs that name that environment, which keeps production credentials away from ordinary test jobs.

You read a secret with the secrets context and, as a habit, pass it to a step through env: rather than pasting it into the command text. The runner then hides exact matches of the value in the log. Variables, read through vars, are not hidden at all, so they are for non-sensitive settings like a region or a colour.

Masking is only a safety net. It matches the exact text, so a changed version of the secret, such as reversed, split or reformatted, is not hidden. A secret inside JSON or YAML text is also less likely to be masked, so keep one value per secret.

OpenID Connect (OIDC) goes further and removes the stored cloud key. The job gets a short-lived signed token that identifies the repository and branch, and the cloud provider decides whether to trust it. Nothing long-lived sits in your secret store to leak. Setting this up differs per provider, so follow your provider's guide.

Try it yourself

Use the ci-demo repository and a fake value only. Never put a real credential in a demo. The GitHub CLI (gh) is used for brevity, and the same settings exist under Settings, Secrets and variables, Actions.

  1. 1
    Create a fake secret and a plain variable
    gh secret set DEMO_TOKEN --body "not-a-real-secret-123"
    gh variable set APP_COLOR --body "blue"

    If you do not use gh, add a repository secret named DEMO_TOKEN and a repository variable named APP_COLOR in the repository settings.

  2. 2
    Read them in a workflow
    cat > .github/workflows/secrets-demo.yml <<'EOF'
    name: secrets-demo
    on:
      workflow_dispatch:
    permissions:
      contents: read
    jobs:
      demo:
        runs-on: ubuntu-latest
        timeout-minutes: 5
        env:
          COLOR: ${{ vars.APP_COLOR }}
        steps:
          - name: Variables are plain text
            run: echo "color is $COLOR"
          - name: A secret is masked when printed whole
            env:
              TOKEN: ${{ secrets.DEMO_TOKEN }}
            run: |
              [ -n "$TOKEN" ] && echo "TOKEN is set, length ${#TOKEN}"
              echo "$TOKEN"
          - name: A changed copy is not masked
            env:
              TOKEN: ${{ secrets.DEMO_TOKEN }}
            run: echo "$TOKEN" | rev
    EOF
    git add .
    git commit -m "Add secrets demo"
    git push
    gh workflow run secrets-demo.yml

    In the log, the second step prints asterisks for the whole value, while the third step shows the reversed characters in clear text. That gap is why you never print secrets, even in a demo.

  3. 3
    Clean up the fake values
    gh secret delete DEMO_TOKEN
    gh variable delete APP_COLOR

    Even a fake secret is better removed once you are done.

What breaks, and where to look

  • A step fails with an authentication error and the token looks empty. There is no clear message that the secret is missing.

    WHY IT HAPPENS
    An unknown secret name evaluates to an empty string. Other causes are a typo in the name, a secret that lives in an environment while the job has no environment: line, or a run from a forked pull request, which does not receive repository secrets.
    CHECK FIRST
    Add a step that runs [ -n "$TOKEN" ] || echo "TOKEN is empty". It tests the value without printing it. Then compare the name and the scope in the settings.
  • A secret value appears in plain text in a run log.

    WHY IT HAPPENS
    Masking only hides exact matches. A transformed copy (reversed, split, base-changed, inside a JSON blob) or a secret that was never registered is not hidden.
    CHECK FIRST
    Treat the secret as leaked. Delete the log or run, rotate the secret at its source, and search workflows for echo, env, printenv and set -x near secret use.
  • A pull request title or branch name runs commands inside your pipeline.

    WHY IT HAPPENS
    A value such as the pull request title was placed directly inside run: with ${{ }}. The text is pasted into the script before the shell starts, so shell syntax in it is executed.
    CHECK FIRST
    Pass untrusted values through env: and use them as quoted shell variables like "$TITLE" instead of expanding them in the command text.

When not to use this

  • Secrets are for credentials only. Use variables for ordinary configuration so people can see and review it.
  • If a provider supports OIDC, prefer it over storing a long-lived key. A key in a secret store still has to be rotated by you.
  • Do not give secrets to workflows that run untrusted code, such as builds of pull requests from strangers.

Check yourself

Why pass a secret through env: instead of writing it into the run: text?

It keeps the value out of the script text, makes quoting safer, and limits the secret to that one step.

Does masking protect a secret that you print after reversing it?

No. Masking matches exact text, so a transformed copy appears in plain text.

What problem does OIDC remove?

The need to store a long-lived cloud credential as a secret. The job gets a short-lived token that the cloud provider validates.

Lesson 51 of 92 in the DevOps path

Search

Search cheat sheets and projects