Metadata-Driven BI
BI's nervous system, made machine-readable
Metadata is BI's nervous system: catalogs, definitions, lineage, metrics and usage — machine-readable, curatable, and driving discovery, reconciliation and reuse instead of tribal knowledge.
What you will learn
- Compose active metadata: catalog, lineage, definitions, usage
- Enable discovery and reconciliation from metadata alone
- Keep metadata curated, not just collected
What is this?
Metadata-driven BI treats descriptions as a first-class asset: a working catalog where every table, report, metric and field carries definitions, lineage, owners, and usage counts — stored as data itself. Discovery runs on it (search 'net revenue' finds the one certified metric), reconciliation reads it (drift checks line against the lineage graph), and reuse compounds because field semantics travel with the data. The enemy is metadata that's collected but never curated.
Why do you need it?
- Tribal knowledge dies with the person who leaves the team.
- Discovery search surfaces the certified metric, not twelve look-alikes.
- Lineage turns a metric shift into a walkable ancestor graph.
- Metadata quality is what separates a catalog from a dump.
Mental model
A library's card catalog vs the shelves of miscellany
Curated metadata
Libraries trust the catalog: the card tells you what's on the shelf, who wrote it, and its shelfmark — and out-of-date cards (stale metadata) are a minefield. Data catalogs are the same: findable, owner-stamped, lineage-linked entries that must be maintained, or the shelves may as well be a dump.
How it works
- Collections: ingest table/report/field metadata automatically where possible.
- Definitions: certified semantics attached to the metric layer end of lineage.
- Lineage: edges recorded — model → metric → report — walked on demand.
- Usage: view/query counts feed discovery ranking and retirement.
- Curation: owner-stamped descriptions, freshness of the catalog itself tested.
Metadata flowing from production to reuse.
Examples
The catalog row that does work
FIELD net_revenueDEF certified semantic layer metric; formula F1 (net excl. tax)OWNER finance-analyticsLINEAGE source.orders -> model.marts -> metric.net_revenue -> report.qbr-one-pagerUSAGE 412 views/30d (rank #3)RECON drift check walks lineage; a redefinition on model.marts flags ALL downstream artifactsSEARCH 'net revenue' returns exactly this metric firstCATALOG FRESHNESS 46/49 entries reviewed <90 daysOne field row powers search, lineage, reconciliation and retirement — metadata earning its keep as data.
In production
- A search that returns the certified 'net revenue' first instead of twelve renamed twins.
- A redefinition on the mart triggers a drift walk to every downstream report.
- Usage ranking retires the 200-views-of-nothing tables politely.
Best practices
- Collect automatically, curate by ownership.
- Attach semantics at the metric layer so definitions travel with lineage.
- Test catalog freshness like pipeline freshness.
- Make discovery rank certified + used over the long tail.
Common mistakes
- A catalog that's a database dump with zero curation.
- Lineage as a static PDF — unfindable the moment things move.
- Definitions duplicated per report instead of traveling from the semantic layer.
- Metadata no one refreshes — 'catalog' becomes lore with timestamps.
Debugging tips
- Search names never find the real metric → certified semantics not attached to discovered fields; fix the metric-layer links.
- Redefinition missed downstream reports → lineage graph incomplete; walk and repair edges.
- Catalog grows stale → no curation cadence; make it a reviewed number, not an event.
Interview checkpoint
Quick quiz
Working metadata is distinguished by:
Catalog freshness should be tested like pipeline freshness.