Data Contracts
The producer-consumer promise, tested
A data contract is the producer-consumer agreement: schema, semantics, freshness, ownership. BI consumers are the customers; contracts turn their trust from hope into a testable promise.
What you will learn
- Define the clauses of a data contract
- Enforce contracts in CI for BI consumers
- Convert contract breaks into incidents
What is this?
A data contract is a written, versioned agreement between a data producer (pipeline, source system) and its consumers (BI models, dashboards, analysts): schema, semantic definitions, freshness SLA, quality checks, and ownership. Enforcement lives in CI — the producer's changes are tested against the contract before they hit consumers, so a BI table that relied on a definition either gets a notice, or the change is blocked. It converts 'please don't break my report' into a machine-checked promise.
Why do you need it?
- BI consumers absorb upstream drift silently until a board number goes quiet.
- A contract gives the consumer a testable guarantee, not a project managers mouth.
- Breaking a contract triggers an incident path instead of a surprise.
- Contract-first design is the emerging standard in modern data stacks.
Mental model
A utility's service-level agreement
The data contract
A utility doesn't promise 'probably fine, most days' — it agrees on voltage, uptime, and what happens when it fails. Your dashboard is the consumer of that service. Data contracts are the supply agreement: the producer signs the schema, semantics and freshness, and the consumer can prove a breach and get it fixed on a path.
How it works
- Clauses: schema (columns, types), semantics (what a column means), freshness, quality (nullability, uniqueness), ownership.
- Publishing: the producer maintains a contract artifact, versioned, browsed by consumers.
- Enforcement in CI: producer changes run against consuming expectations; violations fail the pipeline or require a contract bump.
- Versioning: breaking changes come with a migration notice and a window.
- Breach to incident: contract test failures route to the same incident path as freshness failures.
The contract between pipeline and report.
Examples
A minimal order-events contract
SCHEMA order_id STRING NOT NULL region STRING IN ('EU','US','AP') amount NUMERIC > 0 created_at TIMESTAMPSEMANTICS amount = net order value excl. tax; NULL deniedFRESHNESS <= 30 min lag, daily end-of-day reconciliationQUALITY uniqueness on (order_id), nulls < 0.5%OWNERSHIP producer: orders-team; consumers: BI-mart (analytics)ENFORCE CI gate on producer change; bump = migration windowBREACH routes to incident path (page orders-oncall)Schema, semantics, freshness, quality, ownership, enforcement and breach path — the promise a report can test.
In production
- An orders-table contract gates the producer's schema change before the BI dev_mart breaks.
- A semantic contract (amount = net excl. tax) blocks a silent redefinition that would mislead the revenue model.
- A freshness clause pages the on-call when the daily reconciliation misses its window.
Best practices
- Write contracts from the consumer's promises they actually test.
- Enforce in CI, versioned, with a migration window for breaking changes.
- Anyone may read the contract; only the owner edits it.
- Wire breach paths into incidents, never into 'we'll fix it next month'.
Common mistakes
- Contracts describing intent without enforcement — a PDF with hope.
- Semantics vague (amount = amount) so drift can't be challenged.
- Breaking change shipped with no window and a surprised consumer.
- Contract owned by no one; edits happen at 11pm to 'just make it work'.
Debugging tips
- BI number shifts despite the 'same table' → semantics clause was loose; tighten amount definition.
- Producer kept 'passing' but reports broke → contract not testing the right expectations; verify clause coverage.
- Contract violation ignored → breach routes nowhere; wire the incident path.
Interview checkpoint
Hands-on lab
- Draft the schema and semantics clauses
- Run contract checks in SQL
- Route a contract breach as an incident
Write and Enforce an Orders Contract
50 min · advanced
Quick quiz
A data contract is enforced where?
A breaking data change can ship without a migration window.