Lesson 3 of 7 in DevOps and delivery fundamentals
CI, continuous delivery and continuous deployment
Continuous integration means merging small changes into a shared main line often, with an automated build and tests checking each one. Continuous delivery keeps the main line always ready to release, with a person choosing when. Continuous deployment releases every change that passes the pipeline automatically.
Draft. This lesson is written and waiting for a person to run every command and review it. It is safe to learn from, but check the official documentation for anything you depend on.
Read these first: The delivery lifecycle from plan to monitor
The problem this solves
When people work on separate copies for weeks and merge at the end, the merge is painful and the first real test happens right before release. Bugs from different people's work collide, and nobody knows whose change caused what.
Continuous integration shrinks that moment. Everyone merges small changes frequently, and a machine builds and tests each one in a clean environment. A broken change shows up minutes after it was written, while its author still remembers it.
Delivery and deployment extend the idea past the tests. The artifact that passed is already packaged and ready, so releasing becomes a small, repeatable action instead of a project.
How it works
- People and clients
- Data
- Control
- Runs code
A developer pushes to the repository, which triggers CI to build and test, the artifact is published, a person approves, and it is deployed to production.
- Developer to Repository: 1 push
- Repository to CI server: 2 triggers a run
- CI server to Artifact store: 3 publish if green
- Artifact store to Approval: 4 ready to ship
- Approval to Production: 5 deploy
- Control
- People and clients
- Runs code
In continuous delivery a passing build waits for a manual approval before production, while in continuous deployment it goes straight to production.
- Passing build to Approval: 1 ready
- Approval to Production: 2 yes
- Passing build to Production: 1 automatic
Follow the first picture from the left. A push to the shared repository starts a CI run on a separate machine, which builds the code and runs the tests from a clean state. If everything is green, the pipeline stores the resulting artifact. A red run stops there and the author is told.
CI covers steps 1 to 3. Steps 4 and 5 are delivery and deployment, and the second picture shows the only difference between them. With continuous delivery a person approves the release. With continuous deployment the approval step does not exist, so every green change goes live.
Martin Fowler's description of CI adds some habits that make it work: one shared main line, a build that tests itself, merging at least daily, and treating a broken main line as the first thing to fix.
Try it yourself
You will build a miniature gate. A script plays the CI server and a Git hook runs it before every push, so a bad change cannot reach the shared copy. Linux, macOS or WSL. Everything stays inside ~/devops-lab.
- 1Create a repo and a local stand-in for the shared remote
mkdir -p ~/devops-lab/ci-demo && cd ~/devops-lab/ci-demo git init git config user.name "Learner" git config user.email "learner@example.com" git init --bare ~/devops-lab/ci-remote.git git remote add origin ~/devops-lab/ci-remote.git
A bare repository has no working files and exists only to be pushed to, like a hosted remote. If you rerun this topic, use new folder names or delete the old folders first.
- 2Write the app, the CI script and the hook
echo 'echo 4' > calc.sh cat > ci.sh <<'EOF' set -e echo "ci: syntax check" sh -n calc.sh echo "ci: test" test "$(sh calc.sh)" = "4" echo "ci: all green" EOF printf '#!/bin/sh\nexec sh ./ci.sh\n' > .git/hooks/pre-push chmod +x .git/hooks/pre-push
A
pre-pushhook runs before Git sends anything. If it exits non-zero, the push is cancelled.calc.shshould print the answer to 2 plus 2. - 3Push a good change
git add . && git commit -m "add calc and ci" git push -u origin HEAD
You should see the three
ci:lines first, then Git's normal push messages. - 4Push a bad change and watch the gate stop it
echo 'echo 5' > calc.sh git commit -am "oops" git push
Output stops after
ci: test. The test fails, the hook exits non-zero, and Git prints an error that it failed to push some refs. The remote never received the bad commit. - 5Fix it and push again
echo 'echo 4' > calc.sh git commit -am "fix calc" git push
All three
ci:lines appear and the push goes through. A hook on your laptop can be skipped withgit push --no-verify. A CI server cannot be skipped that way, which is why real teams run the check on a shared server.
What breaks, and where to look
The pipeline fails with a line like
jq: command not found, but the same commands work on your laptop.- WHY IT HAPPENS
- A CI run starts from a clean machine. A tool you installed at some point on your laptop is missing there.
- CHECK FIRST
- Read the first failing line in the CI log, not the last. Add an install step for the tool in the pipeline definition, and pin its version.
A test fails, you re-run the job without changing anything, and it passes.
- WHY IT HAPPENS
- A flaky test. It depends on timing, test order, shared data or the network, so the result is not decided by the code alone.
- CHECK FIRST
- Run just that test in a loop locally, for example
for i in 1 2 3 4 5 6 7 8 9 10; do sh ./ci.sh || break; done. Look for shared state or waits that assume a speed.
CI is green and the feature is still broken in production.
- WHY IT HAPPENS
- Green means the tests passed, not that the product works. Either no test covers the failing path, or production differs from the CI environment.
- CHECK FIRST
- Reproduce the failure, write a test that fails the same way, and compare the CI environment with production settings.
When not to use this
- Continuous deployment without good automated tests and a quick way to roll back ships bugs faster. Continuous delivery with a manual approval is a sound stopping point, especially where audits or regulations require a sign-off.
- A CI server that only builds once a night is a scheduled build, not CI. The value comes from frequent small merges that are verified each time.
Check yourself
What is the difference between continuous delivery and continuous deployment?
Delivery keeps every passing build ready to release and a person decides when. Deployment releases every passing build automatically.
Why does CI run on a separate, clean machine?
It proves the code builds and passes tests without relying on anything that only exists on one developer's laptop.
Why can a pre-push hook not replace a CI server?
A developer can skip a local hook with --no-verify, and each laptop differs. A shared server runs the same checks for everyone every time.
Keep going
Lesson 3 of 92 in the DevOps path