Online Feature Servers
Milliseconds matter when a request is waiting
Online feature serving returns the current value for an entity in single-digit milliseconds — from an in-memory/low-latency store backed by the same definitions as training.
What you will learn
- Describe the online feature serving path and its store types
- Explain how online and offline stay consistent
- Recognize caching and sync failure modes
What is this?
Online feature servers retrieve the current feature value for an entity (a user, transaction, device) quickly enough for synchronous prediction — typically an in-memory or key-value store (Redis, DynamoDB, in-process) synchronized from the offline feature pipeline. They serve the same feature definitions so live values match what training learned.
Why do you need it?
- Real-time inference can't wait for a warehouse query — p99 of a few ms is the requirement.
- Consistent definitions mean the live feature distribution matches training's.
- Online stores also power feature reuse: many models share the same endpoint.
- Async training updated the store in the background, so requests don't compute features.
Mental model
A deli's hot counter stocked from the kitchen
Online store fed by the offline pipeline
Guests order from the counter (online store) — instant. The kitchen (offline pipeline) restocks quietly at night with the same recipes. Nobody cooks per order, and the recipe book never has two versions.
How it works
- The offline pipeline materializes fresh values and pushes them to the online store (nightly, streaming).
- Online store holds entity → current feature values in low-latency storage (Redis/DynamoDB/embedded).
- Servers batch-retrieve features for all entities in a request in one call.
- Cache layers absorb hot entities and reduce store stress.
- Sync lag between offline and online is measured — freshness is an SLA, not an accident.
The online serving path and its sync from offline.
In production
- A fraud scorer fetches 12 features for a transaction entity in one Redis MGET under 3ms p95.
- Feature reads fail open with a default, logged, so a store outage degrades scores instead of erroring.
- A nightly sync refreshes user features; stream updates keep the freshest 24h hot.
Best practices
- Batch multi-entity fetches (pipeline/MGET) — never N round-trips.
- Measure and alert sync lag between offline and online values.
- Define failure semantics: fail-open defaults or fail-closed for high-stakes models.
- Warm caches and rehearse the store on canary before full rollout.
Common mistakes
- One feature lookup per request → latency explosion at p99.
- Online values computed differently than offline (a second implementation).
- No sync lag metering: stale features serve silently for days.
- Treating the online store as a cache of a warehouse and letting it serve stale by design.
Debugging tips
- Slow prediction? Profile feature retrieval: batch anyway, check cache hit ratio.
- Live values != training means? Check sync lag and the feature's materialization job.
- Cold entity (new user)? Backfill defaults per feature and log the miss.
Interview checkpoint
Hands-on lab
- Simulate an offline materialization that writes an online store
- Batch-fetch entity features at request time
- Measure and detect sync lag
Serve Features From a Key-Value Store
25 min · intermediate
Quick quiz
Which store is best suited to synchronous online feature retrieval?
Feature values at serving time are usually computed fresh for every request.