LSTM
Intermediate 15 min read LSTM

LSTM for Time Series Forecasting

Supervised windows over an ordered series

Time-series LSTM forecasting frames the series as a supervised window problem: build lagged windows, train a sequence-to-value model, and evaluate multi-step by re-injection. Baselines first; the LSTM earns its keep only when lags+classical models fall short.

forecastingwindowmulti-stepsequence-to-valuelag featuresrecursive forecast
Prerequisites: LSTM Intuition Time Series
Before we start

What you will learn

  • Frame time-series as windowed supervised learning
  • Build and evaluate one-step and multi-step (recursive) forecasts
  • Run the baseline-first discipline before touching an LSTM
Introduction

What is this?

LSTM forecasting treats the series as a supervised task: each sample is a window of k past values → the next value (or next h values for multi-step). Rolling-window training over a chronological train set teaches the LSTM the conditional next-value mapping. Inference predicts one step and, for multi-step, re-injects predictions — every step after the first is fed by the model itself, so error compounds.

Motivation

Why do you need it?

  • Sequential structure + long history is the LSTM's natural sweet spot vs fixed lags.
  • The recursive-re-feed discipline aligns training-like evaluation with deployment reality.
  • Windows capture seasonality and lagged dynamics the model can condition on.
  • Knowing when NOT to use it (baselines win) saves your team real money.
Think about it

Mental model

Imagine

Piloting a ship by the last k chart marks

Think of

Multi-step forecast by re-injection

Each new plotted position (forecast) becomes the next assumption about where the ship is — errors feed the next step's confidence, so you plan by horizons, not by one perfect point.

Mechanics

How it works

  • Build windows: X[i] = series[i:i+k], y[i] = series[i+k] (one-step) or series[i+k:i+k+h] (multi).
  • Chronological split with a gap: train / val / test — never shuffle.
  • Train the LSTM (sequence-to-value head) on normalized windows with masking.
  • Inference: score the last window; append prediction; slide; recurse for horizons.
  • Evaluate per horizon (MASE/MAE at h=1,2,...) not as a single number.

Rolling window → forecast → re-inject.

Window k
LSTM
ŷ_(t)
Append
Horizons
Real world

In production

  • Energy load, e-commerce demand, and IoT sensor series: hybrid lag-trees + LSTM residual models.
  • Short-horizon fine forecasts where simple regression on lags is too rigid.
  • Multi-horizon plans fed by recursive re-injection with calibrated error bounds.
Guidelines

Best practices

  • Baseline first: naive (last value), seasonal naive, and a lag-based tree/ridge model.
  • Normalize with train-fitted statistics only (never the full series — leakage).
  • Evaluate multi-step by horizons: MAE(1), MAE(3), MAE(7) — not a single mean.
  • Mask padded samples and sort windows by length when length varies.
Watch out

Common mistakes

  • Shuffling the series or normalizing on the test segment — silent leakage.
  • Feeding raw non-stationary levels; differencing/removing trend first changes feasibility.
  • Measuring one-step accuracy and claiming multi-step quality.
  • Ignoring exogenous features (calendar, price, weather) that move the series more than history does.
Root cause analysis

Common issues

Error
Forecast = previous value (identity collapse)
Root cause

Signal too noisy at horizon, or LSTM learned to copy minimal-horizon information

Fix

Add exogenous inputs, different targets (increments), or regularize harder; EDA first.

Error
Multi-step spiral diverges quickly
Root cause

Recursive re-injection amplifies first-step error

Fix

Train direct/h-step heads, cap horizon, add residual/warm-start corrections.

Error
Validation leaks future via normalization
Root cause

Global stats (mean/std of full series) used when scaling

Fix

Fit scaler on train only; transform val/test with those constants.

When things break

Debugging tips

  • Plot forecast vs actual per horizon — errors are beautifully visible as lag/copy artifacts.
  • A 'copy the last value' cul-de-sac symptom = the net learned the naive baseline; add exog/seasonal encoding.
  • Compare LSTM against lag-ridge at equal horizon — the gap quantifies the LSTM's worth.
  • Print the window index vs actual index in the DataLoader — misalignment is a classic off-by-k bug.
Interviews

Interview checkpoint

Practice

Hands-on lab

Window, Train, and Multi-Step Forecast
30 min · intermediate
In progress
Objectives
  • Build supervised windows from a periodic synthetic series
  • Train a small LSTM sequence-to-value forecast
  • Evaluate recursive predictions per horizon against a naive baseline

Window, Train, and Multi-Step Forecast

30 min · intermediate

0/4 steps
0%
sandbox · Linux sandbox
$
Try these commands
Test yourself

Quick quiz

0/2
1

Which evaluation practice is correct for time-series forecasting?

2

A strong lag-based baseline proves you don't need an LSTM.

Go deeper

Official documentation

Forecasting: Principles and Practice (online)
docs