Analytics Engineering
Intermediate 12 min read Analytics Engineering

Documentation and Lineage

Make the warehouse read like a library, not a crime scene

Documentation and lineage turn a warehouse into a readable system: every model explained, every column described, every dependency drawn. New analysts onboard in days instead of forensic archaeology.

documentationlineagedata catalogdata dictionaryonboarding
Prerequisites: Data Modeling Layers
Before we start

What you will learn

  • Write model and column documentation that answers real questions
  • Use lineage to see where a metric came from and what it affects
  • Keep docs attached to code so they update with the models
Introduction

What is this?

Documentation is the human layer of the warehouse: a data dictionary at the model and column level, written where the code lives. Lineage is the auto-generated map of dependencies — what feeds a table and which downstream models it fans out to. Both are produced from the transformation code so they never drift from reality.

Motivation

Why do you need it?

  • A metric with no description is a rumor; a documented metric is a decision.
  • Lineage turns 'this table looks wrong' into one glance at the source of the change.
  • Onboarding analysts stops being a walking tour and starts being a link.
Think about it

Mental model

Imagine

A codebase with no comments and no call graph

Think of

A warehouse with no docs and no lineage

You could read the code (tables), but without a map and notes you cannot safely change anything.

Mechanics

How it works

  • Write a description for every model: what it represents, the grain, who owns it, when it refreshes.
  • Document every column that needs a decision: units, definition, edge cases.
  • Lineage is compiled from ref()/source() relationships automatically.
  • Publish a searchable data catalog from the model files.

Models, docs, and lineage are generated from the same source of truth.

Code
Docs
Lineage
Catalog
Real world

In production

  • A new analyst answers 'what is revenue?' by reading the mart's docs instead of guessing.
  • A schema change check: lineage shows every downstream model that will need attention.
  • Audits trace a disputed metric through lineage to its source rows.
Guidelines

Best practices

  • Document the grain first — everything else hangs off 'one row = one X'.
  • Write column descriptions as decisions (units, definitions), not restatements of the name.
  • Add ownership and freshness to every important model so questions route somewhere.
Watch out

Common mistakes

  • Documenting final reports only, leaving the models that produce them a black box.
  • Column descriptions that read 'the total' — useful documentation admits edge cases.
  • Letting a wiki drift from the code; the docs must live where the models live.
When things break

Debugging tips

  • If docs describe columns that do not exist, the model changed and the YAML key drifted — regenerate.
  • If lineage misses a dependency, the model probably selected from a table by name instead of ref().
Interviews

Interview checkpoint

Practice

Hands-on lab

Document a Model with Lineage
30 min · beginner
In progress
Objectives
  • Write a model-level doc with grain, owner, and refresh cadence
  • Describe the key columns as decisions with units and edge cases
  • Generate the lineage and catalog from the model files

Document a Model with Lineage

30 min · beginner

0/4 steps
0%
sandbox · Linux sandbox
$
Try these commands
Test yourself

Quick quiz

0/2
1

Lineage is drawn from

2

Docs that drift from the code create

Go deeper

Official documentation

dbt documentation
docs
Data lineage (Wikipedia)
docs