Security Engineering
Advanced 17 min read Security Engineering

Compliance as Code

Audit evidence as CI artifacts: policy-as-code, control mapping, continuous evidence, and the difference between 'we claim' and 'we can prove'.

compliance as codepolicy-as-codeinSpeccontrolevidenceSOC 2continuous compliance
Introduction

What is this?

Compliance as code expresses security controls and their evidence as versioned, automated checks — so the auditor sees a pipeline artifact, not a folder of screenshots.

Motivation

Why do you need it?

  • Auditors increasingly accept automated evidence: continuous proof beats a December scramble.
  • Controls expressed as code are testable, diffable, and owned — human essays drift, code re-verifies.
  • It makes compliance a byproduct of engineering discipline rather than a quarter of painful discovery.
Think about it

Mental model

Imagine

A fire inspection checklist

Think of

A building with sensors auto-verifying every rule

The checklist is a moment; the sensors prove the rule continuously, minute by minute.

Mechanics

How it works

  • Policies are written as executable checks (e.g. inSpec/OPA) mapping to framework controls (SOC2, ISO 27001, CIS).
  • Each check produces pass/fail evidence with a timestamp and owner — an artifact, not an anecdote.
  • CI runs the checks on every config change; drift fails the pipeline with an auditable log.
  • Evidence is collected, retained immutably, and exportable for the auditor to sample instantly.

Control to evidence pipeline

Control
Check
Run
Evidence
Audit
Real world

In production

  • An org's SOC 2 control 'intrusion detection' is backed by a check that the SIEM ingestion queue is healthy — automated evidence each day.
  • A bank maps each CIS benchmark item to an inSpec profile and shows the auditor green runs.
  • A SaaS exports a continuous evidence bundle monthly; the audit report is assembled by script, not by PowerPoint.
Guidelines

Best practices

  • Map controls -> checks -> evidence with identifiers that survive re-wording of frameworks.
  • Make evidence immutable and timestamped; an auditor reads the capture, not the claim.
  • Automate the 'evidence refresh' so the gap between current state and the last snapshot is zero.
  • Review the checks themselves as code: a drift in check coverage is a compliance regression.
Watch out

Common mistakes

  • A compliance folder of stale screenshots that nobody re-runs.
  • Checks that pass on a green run but silently stop running — the pipeline is the enforcement.
  • Mapping 'we do MFA' to a check that only tests one system while the fleet has three.
  • Writing policy prose far from the code — the policy and the check contradict each other.
When things break

Debugging tips

  • When a check fails for a 'good' reason, record the exception explicitly with an owner and expiry.
  • Compare the check's coverage list against the control's required scope quarterly.
  • Keep evidence retention in the artifact pipeline; the auditor wants the run history, not a fresh run.
Security

Security considerations

  • Evidence pipelines hold sensitive config views; least-privilege and encrypt the evidence store.
  • An automated control is only as honest as the checks' rules — maintain them like code.
Interviews

Interview checkpoint

Practice

Hands-on lab

Turn a Control into an Automated Check with Evidence
45 min · advanced
In progress
Objectives
  • Pick one control and write an executable check for it.
  • Run it in a loop that writes timestamped evidence.
  • Simulate drift and show the evidence pipeline behaving.

Turn a Control into an Automated Check with Evidence

45 min · advanced

0/4 steps
0%
sandbox · Linux sandbox
$
Try these commands
Test yourself

Quick quiz

0/2
1

The strongest compliance evidence is...

2

A control check that stops running but keeps 'green' last run is...

Go deeper

Official documentation