Hands-on Labs

Guided production scenarios with step-by-step environments.

Stage a Model with a Local Registry

Every model passes through staged gates: development, registration, staging, production and retirement. Stage discipline is what turns a model from an artifact into a governed, auditable production component.

5steps 30 minintermediate

Bundle an Artifact with a Manifest

An ML artifact is the model binary plus everything needed to reproduce and deploy it: code, data pointer, config, metrics and signature. Managing artifacts is how a system becomes replayable instead of precious.

4steps 25 minintermediate

Log Your First MLflow Run

MLflow Tracking is the de-facto open source layer that records runs, metrics and artifacts, and it underpins the rest of the platform — the registry, projects and model serving all hang off tracked runs.

4steps 30 minintermediate

Telemetry: Log Curves and Tags

Parameters are what you set, metrics are what you measure — but only useful metrics are logged per iteration, comparable across runs and honest about the split they came from.

3steps 25 minintermediate

Track Code and Artifacts Together

Metrics promise a result; artifacts deliver the proof. Tracking stores the model binaries, plots and data previews a run produced, linked to the code commit that created them.

3steps 25 minintermediate

Run Comparison with a Gate

Winning an experiment isn't the highest accuracy — it's the model that clears the gates: metric targets, business cost, robustness and simplicity. Comparison is where tracking pays off.

3steps 25 minintermediate

Hash, Diff and Verify a Dataset

Content hashes make datasets machine-verifiable; lineage records the family tree — source, transformations, and what consumed it. Together they make data auditable end to end.

4steps 20 minintermediate

Version a Dataset with DVC

DVC (Data Version Control) stores lightweight pointers in Git and heavy data in remote storage, so datasets, models and pipelines are versioned in the same commit graph as code.

4steps 30 minintermediate

Build a Minimal Dataset Catalog

A data catalog is the searchable front door to your datasets: discoverability, ownership, schema and quality metadata, and the governance that prevents 'which dataset should I use?'

4steps 25 minintermediate

Write a Data Quality Gate

Quality gates are automated assertions on datasets — schema, nulls, ranges, counts, distributions — that run before training or serving trusts the data. Garbage in, gates out.

4steps 25 minintermediate

Make a Run Deterministic

Reproducibility is the property that a model's training pipeline can be re-run against the same inputs and produce the same outcome. It's a discipline: pins, seeds, immutability, and replay.

4steps 25 minadvanced

Wrap a Model with an Enforced Signature

A model signature is the input/output contract a serving endpoint enforces: feature names, types, shapes, and output space. Binaries + signatures turn a model into an API with a schema.

3steps 25 minintermediate

Capture and Query Model Lineage

A model's metadata connects it to its data, code, environment and evaluation evidence; lineage shrinks the graph so any incident traces back to a root cause. Metadata is what makes versions audit-friendly.

4steps 25 minadvanced

Route and Roll Back Between Versions

Rollback is a pointer change when artifacts are immutable and versions are registered; multi-version serving (canary, shadow, A/B) is the infrastructure that makes safe release possible.

4steps 30 minadvanced

Register Models and Transition Stages

MLflow's registry turns tracked runs into governed models: register versions from runs, transition stages, set aliases, add descriptions and control permissions — all through one API and UI.

4steps 30 minintermediate

Enforce Registration Rules

Registering is not enough — you need rules for what evidence a version needs, who may move stages, and how stages map to actual deployments. Stage rules are the policy that keeps the registry honest.

4steps 25 minintermediate

Promote with Aliases and Roll Back

Aliases give semantic names ('champion', 'candidate') to versions so pipelines route on meaning while stage transitions handle governed movement. Together they make promotion describable and reversible.

4steps 25 minintermediate

Trigger a Deployment from Registry State

When deployment reads the registry — image versions, stage state, aliases — release is a registry operation: promotion through stages drives canary, rollout and rollback with no hand-edited config anywhere.

4steps 30 minadvanced

Materialize and Point-in-Time Join

Offline stores materialize historical feature windows for training: versioned, backfillable, and correct at the point in time each label occurred — no future leakage, no awkward joins.

3steps 30 minintermediate

Serve Features From a Key-Value Store

Online feature serving returns the current value for an entity in single-digit milliseconds — from an in-memory/low-latency store backed by the same definitions as training.

3steps 25 minintermediate

Enforce Feature Freshness SLAs

Every feature has a freshness SLA — how current its value must be. Consistency is the guarantee that training and live values mean the same thing. Both are measured, not assumed.

3steps 25 minintermediate

Run a Deterministic Pipeline

Orchestration is the scheduler and dependency manager of ML pipelines: what runs, in what order, on retry, and with what compute. It's the difference between a script and an operation.

4steps 30 minintermediate

Run a Budgeted Random Sweep

Sweeps are orchestrated hyperparameter search: grid, random, Bayesian or bandit. Done right they need budgets, early stopping and honest evaluation — done wrong they're just gaming the tracker.

4steps 30 minintermediate

Simulate Data-Parallel Sync

Distributed training splits work across devices and nodes — data-parallel by default, model-parallel for giants. Your wall-clock gains, losses, and failure semantics all change the moment you scale out.

3steps 25 minadvanced

Replay-Verify a Pipeline

A reproducible pipeline is one whose outputs are pinned to inputs and definition version: same data, code, config and environment → same model. It's the property that makes retrains and audits trustworthy.

4steps 25 minadvanced

Build a Fast ML CI Check

Continuous Integration for ML runs fast, meaningful checks on commits: data schema, code quality, a minimal training smoke test and metric baselines. The goal is catching regressions in minutes, not at retraining time.

4steps 35 minintermediate

Promote a Model Through Stages

Continuous Deployment for ML moves a validated registered model through stages — staging, canary, production — with automated checks and promotable rollback. It's delivery of an artifact, not a rebuild at the edge.

3steps 35 minintermediate

Wire an ML CI Workflow

GitHub Actions gives ML teams a managed CI/CD platform in the code host: workflows for smoke tests, training triggers, and model deployment, all as versioned YAML. Compute-heavy ML still wants runners and caching done deliberately.

4steps 40 minintermediate

Parameterize a Template

Templating turns one forgotten pipeline into a reusable recipe: parameterized steps, shared base images and config-as-code let teams spin up compliant training or serving pipelines without re-inventing the gates.

3steps 35 minadvanced

Gate an LLM Prompt Change

LLMs break classic CI/CD assumptions: prompts, not just weights, are the artifact; evaluation needs evalsets and judges; and cost/latency are checks. CI/CD for LLMs version prompts, weights and evals, and gates on honest outcomes.

3steps 40 minadvanced

Serve a Model Behind REST

The REST API server turns a model file into a request/response service: load the artifact, expose predict endpoints, batch requests, and return the contract alongside the prediction. Small and standard, it's how most models first meet their users.

4steps 35 minintermediate

Package and Serve a TorchServe Model

TorchServe is the PyTorch-native serving engine: handlers, model archives, warm-up and batching built in, plus frontend/backend workers so one instance serves models with concurrency and health built-in.

4steps 40 minintermediate

Model Repository and Batching Config

Triton is the high-throughput inference engine for the heavy lifting: concurrent model serving, dynamic batching, on-the-fly model loading over GPUs, and a performant client contract. It's built for the scale number where REST servers tap out.

3steps 40 minadvanced

Declare a Serving Workload

KServe makes model serving a Kubernetes-native process: an InferenceService CRD with the serving runtime as a configuration. Serving, autoscaling, canaries and rollouts become cluster objects instead of bespoke ops.

3steps 40 minadvanced

Autoscale on the Right Signal

Serving scaling is capacity planning metric-driven: the binding resource (CPU, memory, GPU) sets the ceiling, and autoscaling decisions react to utilization, queue depth and traffic. Autoscale isn't automatic-to-good — it's configured by deliberate signals.

3steps 40 minadvanced

Design a Restartable Batch Job

A batch pipeline is a sequence of bounded, idempotent steps with explicit windows: read → validate → score → write → log. Design decisions are chunking, retry, and output immutability — the details that keep a nightly job honest for years.

3steps 35 minintermediate

Version Prediction Outputs

Batch predictions are only useful if well-written: versioned columns, metadata alongside scores, and a storage layout that consumers read reliably. Output architecture decides whether downstream tools trust the numbers.

3steps 35 minintermediate

Score Windowed Mini-Batches

Mini-batch is the middle ground: process data in small windows (minutes) instead of nightly or per-event, trading a little freshness for throughput. Window semantics — size, watermark, trigger — decide how fresh and how correct the scores are.

3steps 40 minadvanced

Write a Batch vs Real-Time Decision Calculator

Batch vs real-time is a requirements decision: freshness of the answer, cost of the answer being late vs wrong, and operating complexity. A good architecture often mixes both — precompute in batch, personalize in real time.

2steps 35 minintermediate

Score a Stream with State

Streaming inference applies models to a continuous event stream: consume Kafka/streams, join with state, score events with bounded latency and exactly-once-ish semantics. It turns 'react now' from a request into a continuously running process.

3steps 40 minintermediate

Tune Batching for Latency

The serving engine is the performance heart of real-time inference: resident models, batching, GPU scheduling, and optimized runtimes. Choosing and configuring it — TorchServe, Triton, vLLM, engineered REST — decides whether latency budgets actually hold.

2steps 35 minintermediate

Run a Measured Tuning Experiment

Tuning real-time serving means finding the latency/throughput operating point: measure the tail (p95/p99), then adjust batching, concurrency, model precision, and caching in deliberate experiments. Every change trades one axis for another — the discipline is measuring the trade.

3steps 40 minadvanced

Build a Skew Watchdog

Real-time serving inherits governance duties: version registration, auditability of predictions, and guarding against feature skew between training and live. Skew detection and governance hooks close the loop that fast serving otherwise skips.

3steps 35 minadvanced

Declare a Health-Gated Model Deployment

ML pods are the unit of compute: containers with model/training code, resource requests, GPU allocation and probes. Deployments add declarative rollouts and self-healing — the pattern serving and small training both ride on.

3steps 35 minintermediate

Schedule GPUs with Queues and Quotas

GPUs are scarce, indivisible, node-pinned resources. Scheduling them means declaring how many each pod needs, queuing when none are free, and capping teams with quotas so one experiment can't sink the fleet.

2steps 40 minadvanced

Autoscale on a Queue Signal

Autoscaling on K8s spans serving (HPA/KEDA scales replicas on traffic and queue metrics) and batch/training (scale workers and bins). The tricky part is GPU-quantized capacity, model-load warm-up, and scale-to-zero latency tradeoffs.

2steps 35 minadvanced

Declare a Training Job

Training on Kubernetes means Job-based semantics: a training run is a declarative workload with backoffs, checkpoints and log history — single-pod by default, distributed via Kubeflow-style operators (TFJob, PyTorchJob) that handle leader-workers and fault recovery.

3steps 35 minintermediate

Emit Version-Labeled Model Metrics

The cluster generates the three signals — metrics, logs, traces — and ML adds model-level telemetry on top. Observing K8s means wiring Prometheus-style metrics, structured logs and traces while distinguishing platform health from model health.

2steps 35 minadvanced

Build a Monitoring Metric Set

A production model earns a matrix of signals: data drift, prediction drift, latency, error rates, and quality with and without labels. The art is picking the few that signal real problems and pairing them with the thresholds that matter.

3steps 35 minintermediate

Pipeline Telemetry to a Queryable Store

The stack turns prediction events into visibility: emit telemetry, ship it to a store (Prometheus for metrics, a log/index store for events, tracing for request paths), and surface it on dashboards and alerts. MLOps adds model-ready pipelines on top: drift jobs, samplers, and feedback sinks.

3steps 40 minintermediate

Draft and Evaluate a Model SLO

Model SLOs bind monitoring to trust: availability, latency, drift and quality targets that define 'good'. Alerting turns those SLOs into owners with pages on breach — but only when the thresholds, severities and escalation are actually maintained.

2steps 35 minintermediate

Wire Basic LLM Monitoring Signals

LLM monitoring is hard because outputs are open-ended: track safety/guardrail hits, cost and tokens, latency, hallucination proxies, and quality via evals and human feedback — not a single accuracy number. The stack adds judges, evalsets and feedback channels to classic telemetry.

3steps 35 minadvanced

Compute Drift Distances

Detectors compare live and reference distributions: PSI and KS for numbers, chi-square/JS for categories, and model-based tricks for high-dimensional data. No detector is universally 'right' — each trades sensitivity, compute and interpretability.

2steps 35 minintermediate

Detect Numeric Drift

Numeric features drift through mean shifts, scale changes, or tail movements. PSI, KS tests and distribution distances catch them — but binning choices and reference windows determine whether the number you get reflects reality or noise.

2steps 35 minintermediate

Detect Categorical Mix Drift

Categorical drift shows up as new categories, shifted proportions or rare values blooming. Chi-square tests compare observed-vs-expected counts; JS divergence measures distribution distance; the novel-category rate is a standalone red flag.

3steps 35 minintermediate

Run the Drift Triage

A drift alert is a decision point, not a fire: triage (real or artifact?), scope (which features, which segments), then choose — ignore, retrain, rollback or recalibrate. A written playbook makes the decision routine instead of a panicked meeting at 3am.

2steps 35 minintermediate

Diagnose the Drift Family

Data drift moves the inputs; concept drift moves the meaning. Telling them apart matters because the fix differs — re-encode or re-baseline vs retrain on new labels — and many real incidents involve both at once.

2steps 35 minintermediate

Detect Error-Window Drift

Detecting concept drift needs labels or proxies: reconciled outcome error over windows, error-based ADWIN-style tests, or classifier-confidence drift. With no labels, proxies and business outcomes carry the signal — late, but identifiable.

3steps 35 minadvanced

Run a Regime Change Drill

A regime change — a structural shift in the underlying system — is the extreme end of concept drift: the model's world broke. Response needs triage, containment, an accelerated retrain on the new window, and a governance trail — not just 'retrain harder'.

3steps 40 minadvanced

Build a Walk-Forward Gate

Offline evaluation is where you can be strict: time-split validation, backtesting on historical windows, and leakage discipline give repeatable signal — but only if the splits resemble deployment time, not academic random folds.

3steps 35 minintermediate

Score the Live Model

Online evaluation trades offline purity for reality: real predictions face real outcomes, scored on matured labels or outcome proxies when ground truth lags. It's the honest operating metric, full of bias and delay you design around.

3steps 40 minintermediate

Certify and Composite Proxy Signals

Proxy metrics work only if you engineer them honestly: pick signals correlated with the true objective, measure their own drift, and composite multiple proxies so no single noisy signal controls the verdict.

3steps 40 minadvanced

Shadow-Replay the Challenger

Shadow serving runs a candidate on real traffic without affecting outcomes — the gold standard for low-risk evaluation; A/B/A-B testing turns live slices into statistical inference with guardrails and stopping rules.

2steps 45 minadvanced

Craft and Counter an Attack

Adversarial perturbations are small crafted input changes that flip model outputs. Defenses mix pre-processing, robust training (adversarial training, augmentation), and anomaly detection — evaluated by an honest attack-targeted measurement of robustness.

3steps 40 minintermediate

Detect a Poisoned Retrain

Poisoning corrupts the model through its data: label-flips and example injections at train time, or backdoors that activate under secret triggers. Defenses are provenance-first: versioned, access-controlled data, anomaly-screened updates, and honesty metrics that don't trust a promising retrain.

3steps 35 minadvanced

Simulate an Extraction Attack

Extraction steals your model by querying it: adversaries reconstruct a shadow model or recover features/training data. Defenses manage access, detect extraction patterns, watermark outputs, and treat weights as crown-jewel artifacts — not open endpoints.

3steps 40 minintermediate

Profile and Rightsize a Workload

GPU spend is driven by model size, batch efficiency, data movement, and cloud pricing shapes. Attack the real levers: utilization, correctness, and instance-shape matches — not just 'buy cheaper GPUs'.

3steps 40 minintermediate

Elastic Cost Simulator

Autoscaling matches serving and training capacity to actual demand; spot instances buy preemptible capacity at a deep discount. Together they cut idle money — if interrupt tolerance is engineered in.

3steps 35 minintermediate

Profile a Training Loop

Profile before you spend: utilization, memory, data pipeline, and latency footprints—this separates invoices into honest cost levers. Every optimization decision without a profile is astrology with a credit card.

2steps 35 minintermediate

Price Three Platform Blueprints

Most ML platforms are one of a few recognizable shapes: batch-scored, real-time scoring, or LLM-oriented. Choosing the blueprint up front sets the whole architecture, including cost and latency economics.

5steps 30 minintermediate

Simulate an Online Serving Path

The serving tier is where architecture touches users: request routing, feature fetching, batching, health checks and graceful degradation. It is designed around latency distributions, not just averages.

4steps 40 minadvanced

Govern a Model Lifecycle in a Registry

Versioning and governance are what let a platform be audited, rolled back and compliant. The registry is the hub: artifacts, lineage, approvals and every promotion recorded as an immutable event.

4steps 35 minintermediate

Trace a Request Through the Whole Platform

In practice, production ML means wiring every plane together into one operating system. This case study walks an end-to-end platform for fraud scoring — data, training, registry, serving, monitoring — and the incident-driving design decisions behind it.

4steps 40 minadvanced