Data Contracts and Consumer Guardrails
Versioned promises between producer and consumer
A data contract is an explicit, versioned agreement between producer and consumer — shape, semantics, freshness and quality expectations — enforced so producers can't break consumers silently.
What you will learn
- Define the parts of a data contract (schema, semantics, profile, SLA)
- Apply compatibility rules so schema changes never silently break consumers
- Set up a contract registry and rejection workflow for breaking changes
What is this?
A data contract is a machine-readable agreement that describes what producers deliver and what consumers can rely on: schema with types and nullability, semantic meaning of fields, freshness targets, volume bounds and quality checks. Contracts are versioned and stored in a registry. Producers propose changes; if a change breaks backward compatibility, the registry or CI rejects it until consumers are migrated. This moves the social burden ('please don't break my table') into an enforced mechanism.
Why do you need it?
- Reactive breakage is everywhere: a renamed column silently empties a dashboard for days.
- Contracts give producers permission to evolve — as long as the change is non-breaking — without paging every consumer.
- Compatibility rules are deterministic: backward-compatible changes deploy freely; breaking ones need coordination.
- Contracts make quality somebody's job, not nobody's: the producer owns the promise.
Mental model
A published API with a versioned changelog
A data contract
Nobody modifies a public REST API without announcing it; consumers pin versions. Data contracts apply the same discipline to tables and topics: a versioned, documented, enforced interface between teams.
How it works
- Write the contract: schema (fields, types, attributes), semantics (grain, units, timezone), profile (expected volume, freshness), checks (assertions), owner list.
- Store it in a registry (e.g. Schema Registry or a contracts repo) with a schema ID.
- Enforce compatibility: BACKWARD (new schema reads old data), FORWARD or FULL policies on every change.
- Producers submit a change; CI inspects consumers before approving a breaking change.
- Consumers subscribe to a topic/table and are validated against schema + freshness + quality SLAs as part of their own tests.
- On violation: producer is notified, change is blocked, or data is quarantined — never silent.
Producers, registry and consumers in the contract lifecycle.
In production
- A payments platform defines an events contract with BACKWARD compatibility; a producer adds an optional field and deploys without any consumer coordination.
- A data warehouse team monitors freshness and volume SLAs per contract and pings the producer's on-call when a table misses its window.
- A Kafka team reviews proposed schema changes against the consumer manifest file before approving merges.
Best practices
- Start with the 20% of fields consumers truly rely on, not the entire schema.
- Always add optional fields with defaults (backward-compatible), prefix mandatory changes with a v2 topic/table.
- Make the contract the single source of truth the generator reads — don't let code and docs diverge.
- Include a 'deprecation' field: mark fields obsolete N releases before removal.
Common mistakes
- Treating the contract as documentation only — unenforced promises are just wishes.
- Breaking compatibility 'reasonably' and 'just this once' — that's how every silent breakage starts.
- Contracting fields for the future that nobody consumes, bloating review and blocking legitimate change.
- Forgetting semantics: two teams agree on the field name and fight over the definition for weeks.
Debugging tips
- When a consumer breaks, diff the schema hash they validated against vs the current registry version.
- Reproduce the failure by reading old immediately-prior data with the new schema — compatibility is proven at the bytes level.
- Check the registry's rejection log: a repeated rejected change often means the producer team is split on semantics.
- Confirm the contact owner (not just the technical author) is named, so SLA breaches page a person.
Interview checkpoint
Hands-on lab
- Simulate a contract registry with versioned Avro schemas
- Prove an additive-with-default change is backward compatible
- Prove a field-removal change is rejected and blocks the deploy
Enforce Backward-Compatible Schema Changes
30 min · intermediate
Quick quiz
Which change is backward-compatible?
A registry that never rejects breaking changes is effectively documentation.