Lesson 3 of 7 in DevOps and delivery fundamentals

CI, continuous delivery and continuous deployment

Core10 min

Continuous integration means merging small changes into a shared main line often, with an automated build and tests checking each one. Continuous delivery keeps the main line always ready to release, with a person choosing when. Continuous deployment releases every change that passes the pipeline automatically.

Draft. This lesson is written and waiting for a person to run every command and review it. It is safe to learn from, but check the official documentation for anything you depend on.

Read these first: The delivery lifecycle from plan to monitor

The problem this solves

When people work on separate copies for weeks and merge at the end, the merge is painful and the first real test happens right before release. Bugs from different people's work collide, and nobody knows whose change caused what.

Continuous integration shrinks that moment. Everyone merges small changes frequently, and a machine builds and tests each one in a clean environment. A broken change shows up minutes after it was written, while its author still remembers it.

Delivery and deployment extend the idea past the tests. The artifact that passed is already packaged and ready, so releasing becomes a small, repeatable action instead of a project.

How it works

What a CI/CD pipeline does with a push
What a CI/CD pipeline does with a pushA developer pushes to the repository, which triggers CI to build and test, the artifact is published, a person approves, and it is deployed to production.1 push2 triggers arun3 publish ifgreen4 ready to ship5 deployDeveloperRepositoryshared main lineCI serverbuild and testArtifact storetested versionsApprovala person decidesProduction
  • People and clients
  • Data
  • Control
  • Runs code

A developer pushes to the repository, which triggers CI to build and test, the artifact is published, a person approves, and it is deployed to production.

  1. Developer to Repository: 1 push
  2. Repository to CI server: 2 triggers a run
  3. CI server to Artifact store: 3 publish if green
  4. Artifact store to Approval: 4 ready to ship
  5. Approval to Production: 5 deploy
Delivery asks a person, deployment does not
Delivery asks a person, deployment does notIn continuous delivery a passing build waits for a manual approval before production, while in continuous deployment it goes straight to production.1 ready2 yes1 automaticPassing buildcontinuous deliveryApprovala person says goProductionPassing buildcontinuous deploymentProduction
  • Control
  • People and clients
  • Runs code

In continuous delivery a passing build waits for a manual approval before production, while in continuous deployment it goes straight to production.

  1. Passing build to Approval: 1 ready
  2. Approval to Production: 2 yes
  3. Passing build to Production: 1 automatic

Follow the first picture from the left. A push to the shared repository starts a CI run on a separate machine, which builds the code and runs the tests from a clean state. If everything is green, the pipeline stores the resulting artifact. A red run stops there and the author is told.

CI covers steps 1 to 3. Steps 4 and 5 are delivery and deployment, and the second picture shows the only difference between them. With continuous delivery a person approves the release. With continuous deployment the approval step does not exist, so every green change goes live.

Martin Fowler's description of CI adds some habits that make it work: one shared main line, a build that tests itself, merging at least daily, and treating a broken main line as the first thing to fix.

Try it yourself

You will build a miniature gate. A script plays the CI server and a Git hook runs it before every push, so a bad change cannot reach the shared copy. Linux, macOS or WSL. Everything stays inside ~/devops-lab.

  1. 1
    Create a repo and a local stand-in for the shared remote
    mkdir -p ~/devops-lab/ci-demo && cd ~/devops-lab/ci-demo
    git init
    git config user.name "Learner"
    git config user.email "learner@example.com"
    git init --bare ~/devops-lab/ci-remote.git
    git remote add origin ~/devops-lab/ci-remote.git

    A bare repository has no working files and exists only to be pushed to, like a hosted remote. If you rerun this topic, use new folder names or delete the old folders first.

  2. 2
    Write the app, the CI script and the hook
    echo 'echo 4' > calc.sh
    cat > ci.sh <<'EOF'
    set -e
    echo "ci: syntax check"
    sh -n calc.sh
    echo "ci: test"
    test "$(sh calc.sh)" = "4"
    echo "ci: all green"
    EOF
    printf '#!/bin/sh\nexec sh ./ci.sh\n' > .git/hooks/pre-push
    chmod +x .git/hooks/pre-push

    A pre-push hook runs before Git sends anything. If it exits non-zero, the push is cancelled. calc.sh should print the answer to 2 plus 2.

  3. 3
    Push a good change
    git add . && git commit -m "add calc and ci"
    git push -u origin HEAD

    You should see the three ci: lines first, then Git's normal push messages.

  4. 4
    Push a bad change and watch the gate stop it
    echo 'echo 5' > calc.sh
    git commit -am "oops"
    git push

    Output stops after ci: test. The test fails, the hook exits non-zero, and Git prints an error that it failed to push some refs. The remote never received the bad commit.

  5. 5
    Fix it and push again
    echo 'echo 4' > calc.sh
    git commit -am "fix calc"
    git push

    All three ci: lines appear and the push goes through. A hook on your laptop can be skipped with git push --no-verify. A CI server cannot be skipped that way, which is why real teams run the check on a shared server.

What breaks, and where to look

  • The pipeline fails with a line like jq: command not found, but the same commands work on your laptop.

    WHY IT HAPPENS
    A CI run starts from a clean machine. A tool you installed at some point on your laptop is missing there.
    CHECK FIRST
    Read the first failing line in the CI log, not the last. Add an install step for the tool in the pipeline definition, and pin its version.
  • A test fails, you re-run the job without changing anything, and it passes.

    WHY IT HAPPENS
    A flaky test. It depends on timing, test order, shared data or the network, so the result is not decided by the code alone.
    CHECK FIRST
    Run just that test in a loop locally, for example for i in 1 2 3 4 5 6 7 8 9 10; do sh ./ci.sh || break; done. Look for shared state or waits that assume a speed.
  • CI is green and the feature is still broken in production.

    WHY IT HAPPENS
    Green means the tests passed, not that the product works. Either no test covers the failing path, or production differs from the CI environment.
    CHECK FIRST
    Reproduce the failure, write a test that fails the same way, and compare the CI environment with production settings.

When not to use this

  • Continuous deployment without good automated tests and a quick way to roll back ships bugs faster. Continuous delivery with a manual approval is a sound stopping point, especially where audits or regulations require a sign-off.
  • A CI server that only builds once a night is a scheduled build, not CI. The value comes from frequent small merges that are verified each time.

Check yourself

What is the difference between continuous delivery and continuous deployment?

Delivery keeps every passing build ready to release and a person decides when. Deployment releases every passing build automatically.

Why does CI run on a separate, clean machine?

It proves the code builds and passes tests without relying on anything that only exists on one developer's laptop.

Why can a pre-push hook not replace a CI server?

A developer can skip a local hook with --no-verify, and each laptop differs. A shared server runs the same checks for everyone every time.

Lesson 3 of 92 in the DevOps path

Search

Search cheat sheets and projects