Security Engineering In progress
Advanced 17 min read Security Engineering
Compliance as Code
Audit evidence as CI artifacts: policy-as-code, control mapping, continuous evidence, and the difference between 'we claim' and 'we can prove'.
compliance as codepolicy-as-codeinSpeccontrolevidenceSOC 2continuous compliance
Introduction
What is this?
Compliance as code expresses security controls and their evidence as versioned, automated checks — so the auditor sees a pipeline artifact, not a folder of screenshots.
Motivation
Why do you need it?
- Auditors increasingly accept automated evidence: continuous proof beats a December scramble.
- Controls expressed as code are testable, diffable, and owned — human essays drift, code re-verifies.
- It makes compliance a byproduct of engineering discipline rather than a quarter of painful discovery.
Think about it
Mental model
Imagine
A fire inspection checklist
Think of
A building with sensors auto-verifying every rule
The checklist is a moment; the sensors prove the rule continuously, minute by minute.
Mechanics
How it works
- Policies are written as executable checks (e.g. inSpec/OPA) mapping to framework controls (SOC2, ISO 27001, CIS).
- Each check produces pass/fail evidence with a timestamp and owner — an artifact, not an anecdote.
- CI runs the checks on every config change; drift fails the pipeline with an auditable log.
- Evidence is collected, retained immutably, and exportable for the auditor to sample instantly.
Control to evidence pipeline
Controlmapped requirement
Checkexecutable policy
Runon every CI/config change
Evidencesigned, timestamped result
Auditexportable, tamper-evident
Real world
In production
- An org's SOC 2 control 'intrusion detection' is backed by a check that the SIEM ingestion queue is healthy — automated evidence each day.
- A bank maps each CIS benchmark item to an inSpec profile and shows the auditor green runs.
- A SaaS exports a continuous evidence bundle monthly; the audit report is assembled by script, not by PowerPoint.
Guidelines
Best practices
- Map controls -> checks -> evidence with identifiers that survive re-wording of frameworks.
- Make evidence immutable and timestamped; an auditor reads the capture, not the claim.
- Automate the 'evidence refresh' so the gap between current state and the last snapshot is zero.
- Review the checks themselves as code: a drift in check coverage is a compliance regression.
Watch out
Common mistakes
- A compliance folder of stale screenshots that nobody re-runs.
- Checks that pass on a green run but silently stop running — the pipeline is the enforcement.
- Mapping 'we do MFA' to a check that only tests one system while the fleet has three.
- Writing policy prose far from the code — the policy and the check contradict each other.
When things break
Debugging tips
- When a check fails for a 'good' reason, record the exception explicitly with an owner and expiry.
- Compare the check's coverage list against the control's required scope quarterly.
- Keep evidence retention in the artifact pipeline; the auditor wants the run history, not a fresh run.
Security
Security considerations
- Evidence pipelines hold sensitive config views; least-privilege and encrypt the evidence store.
- An automated control is only as honest as the checks' rules — maintain them like code.
Interviews
Interview checkpoint
Practice
Hands-on lab
Turn a Control into an Automated Check with Evidence
45 min · advanced
Objectives
- Pick one control and write an executable check for it.
- Run it in a loop that writes timestamped evidence.
- Simulate drift and show the evidence pipeline behaving.
Turn a Control into an Automated Check with Evidence
45 min · advanced
0/4 steps
0%
sandbox · Linux sandbox
$
Try these commands
Test yourself
Quick quiz
0/2
1
The strongest compliance evidence is...
2
A control check that stops running but keeps 'green' last run is...
Go deeper