Data Warehousing
Intermediate 15 min read Data Warehousing

Staging, Serving and Data Vault Layers

Bronze, silver, gold — and why the raw layer is sacred

Warehouses are layered: raw/staging, cleansed, modeled serving and sometimes data-vault-style staging for auditability. Layers let teams separate concerns and keep raw truth immutable.

stagingservingbronzesilvergolddata vaultmedallion
Before we start

What you will learn

  • Describe the bronze/silver/gold data flow
  • Explain why raw must stay immutable
  • Compare medallion layering with data-vault staging
Introduction

What is this?

Warehouse layering separates transformation stages: bronze (raw snapshots, immutable), silver (cleansed, deduplicated, entity-aligned) and gold (modeled serving for BI). Data-vault adds auditable staging (hubs/links/satellites) for high-uncertainty or regulated sources. Layers make pipelines debuggable, backfillable and traceable.

Motivation

Why do you need it?

  • Immutable raw is the forensic record — any modeled table can be rebuilt from it.
  • Layer separation gives each stage its own failure mode, owner and SLA.
  • Governance can differ by layer: raw is locked, gold is curated and certified.
  • When a 'silver' bug is found, you repair gold by building from raw, not by patching guesses.
Think about it

Mental model

Imagine

Photographic negatives, proofs and prints

Think of

Warehouse layers

Negatives are the untouched original (bronze), proofs are cleaned-up drafts (silver), and the final print arrives at the gallery (gold). If a print is bad, you never retouch a copy of a copy — you go back to the negative.

Mechanics

How it works

  • Bronze/raw: source snapshots and events land verbatim, timestamped, write-once.
  • Silver/curated: dedupe, typecast, conform entities (one customer id), enforce quality tests.
  • Gold/serving: fact/dimension models, semantic views and certified aggregates for BI.
  • Data vault layers stage hubs (entities), links (relationships) and satellites (attributes) for auditable lineage.
  • Each layer has owned refresh jobs; lineage records the path from gold back to bronze.

The layer pipeline and its trusts.

Bronze
Silver
Gold
Consumers
Backfill path
Real world

In production

  • Every source lands in bronze as a full snapshot; silver builds one clean customer per source; gold models orders, revenue and churn for dashboards.
  • A regulated bank keeps a data-vault style audit trail—every change to a customer attribute is a satellite row with valid_from/valid_to.
  • When a gold aggregate is wrong, the team backfills from silver in minutes, then diff-reconciles.
Guidelines

Best practices

  • Treat bronze as append-only; never let a transform clobber a snapshot.
  • Put quality gates between layers: silver should pass expectations before gold builds.
  • Give each layer an owner and prefix (bronze_*, gold_*) so lineage is self-describing.
  • Document the backfill procedure per layer — restore is a feature, not an afterthought.
Watch out

Common mistakes

  • Letting reports read silver or bronze 'because it's faster' — definitions fork.
  • Updating raw in place when a source re-emits, corrupting forensic history.
  • Skipping silver and modeling from bronze repeatedly, spreading messy logic.
  • Building four layers because the vendor sells four badges, not because you need them.
Root cause analysis

Common issues

Error
Two gold reports disagree
Root cause

One report bypassed gold and read silver

Fix

Enforce gold-only access for consumers; keep silver internal.

Error
Backfill can't reproduce a table
Root cause

Raw was overwritten instead of appended

Fix

Make bronze append-only with snapshot timestamps; rebuild from history.

When things break

Debugging tips

  • Track the layer a discrepancy entered: diff bronze→silver→gold counts to localize.
  • If gold is right but a silver table diverges, check the silver quality gates.
  • Lineage from gold to bronze tells you which raw wrote the bad value.
Interviews

Interview checkpoint

Practice

Hands-on lab

Layered Warehouse in SQLite
40 min · intermediate
In progress
Objectives
  • Land raw snapshots into bronze
  • Curate entities into silver
  • Build a gold serving model and backfill it

Layered Warehouse in SQLite

40 min · intermediate

0/5 steps
0%
sandbox · Linux sandbox
$
Try these commands
Test yourself

Quick quiz

0/2
1

Which layer should be strictly immutable?

2

Silver is an optional shortcut that reports can safely bypass.

Go deeper

Official documentation