ML Fundamentals
Intermediate 14 min read ML Fundamentals

The Machine Learning Workflow

From notebook to monitored loop

A production workflow is a loop: define the business metric, build the baseline, engineer features, train with experiment tracking, deploy with monitoring, and retrain on evidence — not a one-shot notebook.

ML workflowdeploymentmonitoringretrainingexperiment trackingMLOps
Prerequisites: Model Selection and Tuning
Before we start

What you will learn

  • Sketch the full ML lifecycle: framing, data, modeling, deploy, monitor, retrain
  • Define deployment and monitoring basics (versioning, drift, champion/challenger)
  • Explain why MLOps is a process problem, not just a tooling problem
Introduction

What is this?

The ML workflow is the end-to-end loop that turns a business question into a maintained model. Phases: framing (metric + cost of errors), data (labels, splits, lineage), experiments (tracked and reproducible), deployment (blessed artifact + serving), monitoring (drift, quality, business impact), and retraining (scheduled or on-evidence). A notebook is a sketch; production is the plumbing around it.

Motivation

Why do you need it?

  • Most real failures are in serving, versioning and drift, not in the model training step.
  • Type 'it works in my notebook' nowhere in the career search — reproducibility is the difference between ML and ML practice.
  • Monitoring closes the loop: without it, a good model quietly rots as the world changes.
  • Audited, versioned pipelines earn stakeholder trust and regulatory pass.
Think about it

Mental model

Imagine

A restaurant: cooks, pantry, service line, waitstaff feedback

Think of

The ML workflow

Cooking (training) is one station. A great restaurant survives because the pantry (data), the line (deployment), and the feedback to the kitchen (monitoring) all work as a loop.

Mechanics

How it works

  • Frame -> metric + baseline: define success and what 'worse' costs the business.
  • Data -> lineage and validity: record source, splits, and validation gates.
  • Experiment -> versioned: each run logs code, data hash, config, metrics.
  • Deploy -> serve: bless one artifact; champion/challenger gates for rollout.
  • Monitor -> drift and quality: feature/target drift, accuracy telemetry, alert thresholds.
  • Retrain -> scheduled or triggered, and the loop restarts with fresh framing.

The production ML loop.

Frame
Data
Experiment
Deploy
Monitor
Retrain
Hands-on

Examples

A minimal tracked pipeline sketch

A minimal tracked pipeline sketch
import hashlib
import json
import numpy as np
from sklearn.datasets import make_classification
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score
 
X, y = make_classification(n_samples=2000, n_features=8, random_state=0)
data_hash = hashlib.sha256(X.tobytes() + y.tobytes()).hexdigest()[:10]
name = "churn_v2"
cfgs = [
    {"C": 0.1, "solver": "liblinear"},
    {"C": 1.0, "solver": "liblinear"},
]
 
for i, c in enumerate(cfgs):
    pipe = make_pipeline(StandardScaler(), LogisticRegression(random_state=0, **c))
    score = cross_val_score(pipe, X, y, cv=4, scoring="roc_auc").mean()
    run = {"name": name, "config": c, "data_hash": data_hash, "cv_auc": round(score, 4)}
    print(json.dumps(run))

Every run records its data hash, config and metric to a JSON line — the seed of an experiment tracker: reproducibility and comparison without a shared notebook.

Real world

In production

  • Banking: every model artifact ships with a model card (training data hash, splits, metrics, bias notes) for audit.
  • E-commerce ranking: champion/challenger rollout lets a new model prove itself on 5% traffic before full share.
  • SaaS ops: drift alerts on feature distributions fire before the accuracy metric even moves.
Guidelines

Best practices

  • Decree a single source of truth for data splits (a function, not copy-paste); log every training run; bless artifacts with checksums; monitor both performance and data drift; make retraining a decision, not a cron job.
Watch out

Common mistakes

  • No versioning of data or code, so a model can't be reproduced; monitoring only accuracy while features drift silently; retraining on a schedule regardless of drift; one-off notebooks as the 'pipeline'.
When things break

Debugging tips

  • When a model looks good in QA but bad in prod, diff the train/serve feature pipelines first.
  • Attach a data-hash check at serving time so it's obvious when a table changed.
  • Alert on prediction-volume or score-distribution shifts — they precede accuracy collapse.
Interviews

Interview checkpoint

Test yourself

Quick quiz

0/2
1

Why version the DATA hash, not just model code?

2

Drift triggers an alert. First action?

Go deeper

Official documentation

The ML lifecycle (see sklearn/Google guides)
video
MLflow: open-source ML lifecycle
docs
MLOps maturity model | Wikipedia
docs