dbt
Intermediate 13 min read dbt

dbt Sources and Freshness

Name the raw world once, monitor it forever

dbt sources declare where raw data lives, so models can read it with source() and dbt can check that it keeps arriving. Freshness checks turn 'the pipeline died' into an alert on the right page.

dbtsourcessource()freshnessloaded_at_fieldmonitoring
Prerequisites: dbt Models
Before we start

What you will learn

  • Declare a source and its tables in YAML
  • Read source tables in models with source()
  • Configure freshness thresholds and run source freshness
Introduction

What is this?

A dbt source is a declared view of a raw table (or a whole source system) in YAML. Models read sources with source('raw', 'orders') instead of hard-coded names, and dbt tracks their freshness: how recent the newest loaded_at row is, compared to warn and error thresholds.

Motivation

Why do you need it?

  • Declaring sources separates raw reality from your cleaned models.
  • Freshness turns silent upstream stalls into paged alerts.
  • source() keeps the dependency graph honest — lineage includes the raw world.
Think about it

Mental model

Imagine

A labeled crate from the supplier

Think of

A dbt source

The crate (source) is declared, labeled with its freshness, and the kitchen never worries about what is inside — they just call the crate's name.

Mechanics

How it works

  • Declare source schema: name, tables, columns, and loaded_at_field.
  • Reference with source('name', 'table') in model SQL.
  • dbt source freshness queries max(loaded_at_field) and compares to thresholds.
  • Warn and error alerts fire on freshness breaches in CI or scheduled runs.

Sources feed models and freshness checks watch their pulse.

Load
Declare
Consume
Monitor
Real world

In production

  • The nightly revenue pipeline alerts when the orders source stops updating.
  • A new analyst sees staging models back-linked to real raw tables via lineage.
  • Freshness thresholds differ per system: payments strict, CRM lenient.
Guidelines

Best practices

  • Declare every raw table you consume so nothing is read unadvertised.
  • Pick loaded_at_field as the earliest reliable arrival timestamp.
  • Set warn tight and error loose, and page owners by freshness threshold.
Watch out

Common mistakes

  • Hard-coding the raw schema.table name in models instead of source().
  • Setting freshness on a field that is not the load timestamp, so reports look stale.
  • Forgetting to schedule dbt source freshness to actually run.
When things break

Debugging tips

  • If freshness warns daily, the loaded_at field may not be the ingestion timestamp.
  • If source() fails, the source name or table is missing from the YAML definition.
Interviews

Interview checkpoint

Practice

Hands-on lab

Declare Sources and Watch Freshness
30 min · intermediate
In progress
Objectives
  • Declare the raw source and its tables in YAML
  • Rewrite a model to use source()
  • Configure and run freshness checks

Declare Sources and Watch Freshness

30 min · intermediate

0/4 steps
0%
sandbox · Linux sandbox
$
Try these commands
Test yourself

Quick quiz

0/2
1

Reading a raw table in dbt is done via

2

Freshness checks query the

Go deeper

Official documentation

dbt sources
docs
source freshness
docs