Data Quality
Advanced 16 min read Data Quality

Data Quality Observability and DQ as Code

Quality you can measure, trend and page on

Treat data quality like a production service: metrics, SLIs/SLOs, alerting, on-call and CI — all captured as data-quality-as-code with history, trends and ownership.

DQ as codeobservabilitySLISLOhistoryon-callCI/CD
Prerequisites: Anomaly Detection for Data Data Expectations and Assertions
Before we start

What you will learn

  • Define data-quality SLIs and SLOs per dataset (freshness, completeness, profile score)
  • Make checks part of CI and deploys (DQ as code), not ad-hoc scripts
  • Build a feedback loop: alerts → on-call → runbook → improved checks
Introduction

What is this?

Data-quality observability borrows from SRE: define measurable SLIs (freshness lag, completeness %, uniqueness %, profile drift), wrap them in SLOs with error budgets, and monitor history so you can trend, not just trip. 'DQ as code' means checks live in repos alongside pipelines, run in CI before merges, are versioned and reviewed, and have owners and runbooks — the same lifecycle as application code. The sum is quality you can trust because you can page, trend and improve it.

Motivation

Why do you need it?

  • Ad-hoc checks die the moment their author leaves; checked-in code survives reviews and rotation.
  • SLIs with error budgets quantify risk ('3 breaches left this month') instead of guessing.
  • Historical trend separates 'getting worse' from 'stable, briefly noisy' — the question every stakeholder asks.
  • Owning on-call for data quality changes behavior: engineers fix sources to stop pages.
Think about it

Mental model

Imagine

A kitchen with an HACCP logbook and a health inspector

Think of

Data quality observability

Good kitchens log temperatures and cleanliness on a schedule (history), set corrective-action limits (SLOs), and get inspected — the logbook forces steady behavior. DQ observability is that discipline, applied to tables.

Mechanics

How it works

  • Define SLIs per dataset: freshness lag, completeness %, uniqueness %, row volume, profile (median/percentiles drifts).
  • Set SLOs + error budgets: e.g. 99% of runs fresh within 10 min per rolling 30 days.
  • Check-in quality code: expectations, thresholds, SQL — in the pipeline repo, reviewed like app code.
  • Run checks in CI on every pipeline change and on a schedule in production.
  • Collect history into a store; compute trends, seasonally-adjusted baselines and breach rates.
  • Wire alerts to ownership and runbooks; every page terminates in a check improvement.

Closed loop: define → check → measure → alert → improve.

Define
Check
Measure
Alert
Improve
Real world

In production

  • A unicorn analytics org sets a 30-day budget of 3 freshness breaches per table; a breach now costs the team its page, so producers fix the source feed instead of patching the report.
  • A bank runs data quality checks in the same CI as its dbt models: merging a model that violates a new uniqueness test blocks the deploy.
  • A SaaS vendor exposes a public quality score per dataset to customers, computed from the same SLIs they monitor internally.
Guidelines

Best practices

  • Adopt SLI/SLO language so quality is negotiated, not imposed: owners set budgets.
  • Store every check result with a run id; reproducibility beats a pretty dashboard.
  • Automate runbook generation from check metadata (owner, fix steps per assertion).
  • Start trend-over-threshold: meet weekly to review top dataset scorecards, not only the pages.
Watch out

Common mistakes

  • Building a dashboard nobody owns — observability without ownership is decorating a corpse.
  • Not versioning checks, so a 'fix' in an environment nobody can reproduce later.
  • Running checks only after deploy, missing the pre-merge gate entirely.
  • Treating every page as noise instead of feeding it back into the checks (the loop is the point).
When things break

Debugging tips

  • First check the trend: a flat scorecard with one spike is 'noise'; a monotonic decline is 'incident mode'.
  • When a page fires, open the runbook linked from the alert — it should name the fix and owner.
  • Re-run yesterday's failed check against today's data: if it passes, the issue was a transient run, not data.
  • Compare the check version that fired vs the current repo head to spot a stale rollout.
Interviews

Interview checkpoint

Practice

Hands-on lab

Stand Up Data-Quality SLIs with SQLite
30 min · intermediate
In progress
Objectives
  • Compute the core SLIs: freshness lag, completeness %, uniqueness %
  • Store daily SLI history and compute a 30-day breaching rate (error budget)
  • Emit a dashboard-friendly scorecard per dataset

Stand Up Data-Quality SLIs with SQLite

30 min · intermediate

0/4 steps
0%
sandbox · Linux sandbox
$
Try these commands
Test yourself

Quick quiz

0/2
1

Which practice is the defining feature of 'DQ as code'?

2

An SLO with an error budget translates quality into a quantified, negotiable risk.

Go deeper

Official documentation

SRE handbook: monitoring and alerting
docs
dbt tests in CI
docs