Lesson 4 of 8 in CI/CD pipelines
Caching and faster pipelines
A cache stores files from one run, such as downloaded dependencies, so that later runs can restore them instead of fetching them again. A cache is identified by a key, and the key decides when old contents are reused and when they are replaced.
Draft. This lesson is written and waiting for a person to run every command and review it. It is safe to learn from, but check the official documentation for anything you depend on.
Read these first: Build, test and artifacts
The problem this solves
Every job starts on an empty machine, so every run downloads the same dependencies again. On a project with many packages that is a large share of the total run time, and it repeats on every push.
Caching removes the repeated download. The risk is stale data: if the cache is reused when it should have been rebuilt, the pipeline tests old dependencies. A good key is the thing that prevents that.
How it works
- Runs code
- Other
- Data
The job hashes a lockfile into a key and looks it up in the cache store. A hit restores the files, and a miss runs a fresh install whose result is saved at the end of a successful job.
- Job starts to Cache key: 1 hash lockfile
- Cache key to Cache store: 2 look up key
- Cache store to Restored files: 3a hit: restore files
- Cache key to Fresh install: 3b miss: install
With actions/cache, you give a path to save and a key. If a cache with exactly that key exists, it is a hit and the files are restored. If not, it is a miss, your steps run normally, and a new cache is saved when the job ends successfully.
A cache cannot be edited after it is saved, so the way to update it is to change the key. That is why keys usually include a hash of the lockfile, for example hashFiles('package-lock.json'). When the dependencies change, the hash changes, the old cache is not matched and a new one is built.
restore-keys lists fallback prefixes. If there is no exact match, the most recent cache whose key starts with one of them is restored. That is faster, but it can restore slightly old files, so your install step must still run afterwards.
Caches are scoped to branches. A run can use caches from its own branch and from the default branch, but not from a sibling branch. Caches that nobody reads for 7 days are removed, and there is a per-repository size limit.
Many setup actions, such as actions/setup-node with a cache option, build this key for you. A second speed-up is concurrency with cancel-in-progress: true, which cancels an older run on the same branch when a newer push arrives. Use that for tests, not for deployments.
Try it yourself
Use the ci-demo repository. The cached content here is a timestamp file, so you can see whether it came from the cache or was rebuilt.
- 1Add a workflow that caches a folder
echo "one 1" > deps.txt cat > .github/workflows/cache-demo.yml <<'EOF' name: cache-demo on: push: branches: [main] permissions: contents: read concurrency: group: cache-demo-${{ github.ref }} cancel-in-progress: true jobs: demo: runs-on: ubuntu-latest timeout-minutes: 5 steps: - uses: actions/checkout@v4 - id: cache uses: actions/cache@v4 with: path: .demo-cache key: demo-${{ hashFiles('deps.txt') }} - name: Build the cache contents if: steps.cache.outputs.cache-hit != 'true' run: | sleep 20 mkdir -p .demo-cache date > .demo-cache/built-at.txt - name: Show what is in the cache run: cat .demo-cache/built-at.txt EOF git add . git commit -m "Add cache demo" git pushOn the first run there is no cache, so the build step runs and takes about 20 seconds. The
cache-hitoutput is onlytrueon an exact key match. - 2Run again with the same deps.txt
git commit --allow-empty -m "Rerun with the same key" git push
The key is the same, so the cache is restored and the build step is skipped. The last step prints the same timestamp as before, which proves the file came from the cache.
- 3Change the dependency file
echo "one 2" > deps.txt git commit -am "Change deps" git push
The hash of
deps.txtchanged, so the key changed, the lookup misses and the build step runs again with a new timestamp.
What breaks, and where to look
The log says
Cache not found for input keyson every run, even when nothing changed.- WHY IT HAPPENS
- The key contains something that changes each run, such as the run id or a date, or
hashFiles()points at a path that does not match any file. - CHECK FIRST
- Print the computed key in an earlier step and compare it across two runs. Confirm the file named in
hashFiles()exists at that path.
You fixed the cached files, but later runs keep using the old broken ones.
- WHY IT HAPPENS
- An existing cache cannot be changed. As long as the key stays the same, the old contents are restored every time.
- CHECK FIRST
- Change the key, for example by adding a version prefix such as
v2-, or delete the old cache from the repository's Actions caches page.
A new feature branch never finds the cache that another feature branch just created.
- WHY IT HAPPENS
- Caches are only visible to the same branch and the default branch, not to sibling branches.
- CHECK FIRST
- Make sure the default branch also builds and saves the cache, so every branch can fall back to it.
When not to use this
- Do not cache build outputs you want to trust. A cache is a speed-up and not a source of truth, and a poisoned cache would silently feed bad files into later runs.
- If the install takes only a few seconds, a cache adds complexity for no gain. Measure first.
Check yourself
Why does the cache key usually include a hash of the lockfile?
Caches cannot be edited, so a changing key is how stale dependencies are replaced when the lockfile changes.
What is the difference between a cache and an artifact?
An artifact carries output between jobs of one run and is for you to download. A cache is a speed-up for later runs and may be missing at any time.
What does cancel-in-progress: true do?
It cancels an older run in the same concurrency group when a newer run starts.
Keep going
Lesson 50 of 92 in the DevOps path