Time Series with pandas
Time-aware frames: trends, windows, seasonality
Datetime indexes unlock resampling, rolling windows, lagged features and cohort timestamps — the machinery of month-over-month, trends, seasonality and 'same time last year'.
What you will learn
- Parse and index dates, set a DatetimeIndex
- Resample to weekly/monthly grains
- Compute rolling, shift and period-over-period comparisons
What is this?
A DatetimeIndex makes the frame time-aware: resample('M').sum() rebins daily to monthly, rolling(7).mean() adds trailing windows, shift(1) creates lag columns for period-over-period deltas. Parsing happens at read (parse_dates) or with pd.to_datetime. This is the skeleton under every MoM, YoY and cohort table.
Why do you need it?
- Every trend chart is a time-series operation under a line plot.
- MoM/ZoZ/7-day-avg are resample and rolling, correctly defined once.
- Lag columns (shift) power seasonality and period comparisons without SQL gymnastics.
- Cohort retention is fundamentally a time-series reshape of first-event vs later windows.
Mental model
A tide station's logbook, minute by minute
A time-indexed DataFrame
The station records every reading with a timestamp; the port office resamples to hourly/daily averages, computes a 7-day rolling mean to smooth storms, and compares today against yesterday's lag to spot tides — all because the logbook is indexed by time.
How it works
- pd.to_datetime(col) parses a column; setting df.index = pd.DatetimeIndex(...) enables time methods.
- .resample('W').sum() / .resample('M').mean() rebin the data to weekly/monthly.
- .rolling(window=7, min_periods=1).mean() computes trailing windows with a sensible early cutoff.
- .shift(1) on a sorted series creates the previous value's row — for period-over-period deltas.
- .dff() computes exact first-differences; .pct_change() gives percent change.
- Grouping by month or week name is a groupby(pd.Grouper(freq='M')) alternative.
Time-aware pipeline.
Examples
A day's worth of data → weekly trend
import pandas as pdimport numpy as np idx = pd.date_range("2026-06-01", periods=42, freq="D")s = pd.Series(np.random.default_rng(1).normal(100, 10, 42), index=idx)print(s.resample("W").mean().round(1).head())print(s.rolling(7).mean().iloc[-1].round(1))print(s.pct_change().iloc[-1].round(3))resample yields weekly means, rolling gives a trailing 7-day average, and pct_change produces the latest day-over-day change.
In production
- A revenue report computes weekly resamples and a 4-week rolling average for trend lines.
- A demand planner builds lag=28 features for order forecasting and detected seasonality.
- A support team measures that 7-day rolling mean response time versus the SLA line.
Best practices
- Sort by the time index before shift/rolling — unsorted windows are fabricated.
- Use min_periods to keep early windows honest.
- Choose the grain at the question level: daily for volatility, weekly/monthly for trends.
- Store UTC and convert to display timezone once, at the end.
Common mistakes
- Calling rolling on an unsorted frame and getting scrambled averages.
- Resampling before sorting/NaN-trimming, contaminating the bins.
- Comparing 'this month' to 'last month' across uneven day counts.
- Mixing UTC storage with local display anywhere mid-pipeline.
Debugging tips
- A rolling mean jumps oddly → check index order and gaps (asfreq to make missing days visible).
- shift deltas look wrong → confirm the index frequency with .freq.
- resample throws on non-datetime index → you forgot parse_dates/to_datetime.
- Plot before/after resample to eyeball whether the grain step destroyed the signal.
Interview checkpoint
Hands-on lab
- Parse timestamps and set a DatetimeIndex
- Resample daily rows to weekly totals
- Compute rolling averages and week-over-week change
Build a Weekly Revenue Trend
30 min · intermediate
Quick quiz
resample('M').sum() operates on:
shift(1) requires the frame to be sorted by the index first.