Batch vs Real-Time Decision
Choosing, and often mixing, delivery models
Batch vs real-time is a requirements decision: freshness of the answer, cost of the answer being late vs wrong, and operating complexity. A good architecture often mixes both — precompute in batch, personalize in real time.
What you will learn
- Score decision criteria for batch vs real-time
- Estimate the cost of a late answer vs a wrong answer
- Design hybrid systems that precompute and personalize
What is this?
The batch-vs-real-time decision weighs how fresh an answer must be (freshness), how costly lateness or staleness is (cost-of-late), and the operating complexity (ops). It's rarely either/or — hybrid architectures precompute the heavy parts in batch and answer interactive queries in real time.
Why do you need it?
- Building real-time when freshness isn't needed burns money and ops for nothing.
- Running nightly when the decision matters in the moment causes wrong rulings.
- The same model frequently serves both: precomputed recommendations + live rerank.
- Choosing deliberately prevents the 'we always do it this way' trap.
Mental model
Weather forecast: bulk nightly models + live radar updates
Hybrid batch/real-time
Long-range forecasts (batch) refresh nightly and are fine for planning; live radar and storm alerts (real-time) update the moment conditions change. They're the same discipline running at two cadences, and both get consumed.
How it works
- Score the workload: latency budget, freshness requirement, variance of demand, cost sensitivity.
- Batch covers the precomputable, expensive, low-urgency portion.
- Real-time covers the interactive, high-urgency, low-volume portion.
- A shared feature/model version keeps both paths consistent.
- Serving layers mix precomputed results with live inference where latency allows.
Choosing and combining both paths.
In production
- A retailer recomputes catalog rankings nightly, then adds real-time personalization per request.
- Fraud: nightly model refresh in batch, sub-second scoring at checkout.
- A news app precomputes article scores hourly and live-ranks breaking items.
Best practices
- Start from requirements — latency budget and freshness need — not stack preference.
- Quantify cost-of-late (missed recommendation) vs cost-of-wrong (bad ruling) before deciding.
- Keep feature/model versions identical across both paths to avoid train-serve skew.
- Design the hybrid boundary: what precomputes, what re-rank runs live.
Common mistakes
- Assuming 'modern' means real-time for everything.
- Nightly scores used for a decision that's only valid at this moment.
- Duplicating logic between batch and live paths that then drift.
- Buying a streaming platform where a cron suffices.
Debugging tips
- Hybrid answers inconsistent — diff the batch vs live path's features and model versions.
- Latency target unmet — the 'batch precompute' may be ignoring the heavy interaction the live path redoes.
- Nightly scores stale for users — raise the batch cadence or move the interactive part live.
Interview checkpoint
Hands-on lab
- Score workload criteria across batch and real-time
- Quantify cost-of-late vs cost-of-wrong
- Emit a recommendation including hybrid options
Write a Batch vs Real-Time Decision Calculator
35 min · intermediate
Quick quiz
The first thing to pin down before choosing batch or real-time is…
A system that both precomputes and answers live is contradictory.