The ML Pipeline in Brief
The map of the whole field
Machine learning turns data into a trained model that predicts on new inputs. The pipeline — data, features, training, eval, serving — is the map every ML and AI engineering conversation uses.
What you will learn
- Name the stages of an ML pipeline and their outputs
- Explain where trained-from-scratch ML and LLM applications diverge
- Spot the parts AI engineers own vs delegate to model providers
What is this?
Machine learning is teaching a program to map inputs to outputs by example rather than by hand-written rules. The standard pipeline: collect data → engineer features → split train/test → train a model → evaluate → serve. AI engineering uses the same pipeline for models like classifiers, but for LLM apps most of the heavy training is rented from providers — your pipeline focuses on data, retrieval, prompting, evaluation and serving.
Why do you need it?
- You can't manage an LLM app without knowing where its behavior actually comes from.
- Evaluation, data quality and serving are the stages AI engineers truly own.
- Language like 'features', 'overfitting' and 'metrics' is the shared vocabulary across teams.
- When a smarter model isn't enough, it's usually a data or evaluation problem — which the pipeline makes visible.
Mental model
Learning to cook from labeled recipe photos
Machine learning from labeled data
You don't memorize rules; you see many examples and yet you infer the pattern. ML is the same but mechanical: from many (input, answer) pairs, the model infers a mapping that generalizes to unseen inputs.
How it works
- Data collection: gather examples of the input and the expected output.
- Feature engineering: turn raw data into numbers the model consumes — a text becomes a vector, an image a tensor.
- Split: hold out part of the data (test) the model never trains on, to estimate real-world performance.
- Training: the optimizer adjusts model parameters so predictions on training data improve.
- Evaluation and serving: measure on held-out data, then ship the model behind an API.
The five stages of a classic ML pipeline.
In production
- An ML team builds a fraud classifier: transaction features, train/test split by time, logistic regression, AUC-gated rollout.
- An AI app evaluates a RAG pipeline offline with golden tests — the 'evaluate' stage of an LLM pipeline.
- A search team tunes a re-ranking model on click data, retraining monthly as user behavior drifts.
Best practices
- Always hold out data you will never train on — leaks inflate confidence dangerously.
- Evaluate on the problem metric, not just loss: precision/recall/NLG quality as the product cares.
- Add a simple baseline before a complex model; if the baseline wins, the data is the problem.
- Version data, features and models so a regression is attributable.
Common mistakes
- Tuning hyperparameters on the test set until the test set stops meaning anything.
- Downstream consumers shipping a model without knowing the eval metric or its plateau.
- Skipping the baseline and celebrating gains that random guessing already had.
Common issues
Excellent test scores, bad production behaviorData leakage or train/test distribution shift
Re-split cleanly, keep test data pristine, and monitor drift in production.
Model looks great offline but is never adoptedOffline metric doesn't match the product metric
Define the business metric first and eval against it.
Debugging tips
- Weird predictions usually trace to data quality or feature leaks before model tuning.
- Compare train vs test performance to detect overfitting before customers do.
- Split by time for time-series (fraud, churn) — random splits leak the future.
Interview checkpoint
Quick quiz
Which stage estimates how a model will do on real, unseen data?
AI engineering only means calling a model API with a prompt.