Guided production scenarios with step-by-step environments.
A vector is an ordered list of numbers; the dot product measures how aligned two vectors are and is the fundamental operation behind predictions and similarities alike.
A derivative is a rate of change; the gradient generalizes it to many inputs. Gradient descent walks downhill on the loss surface, which is how nearly every ML model learns.
Descriptive statistics summarize a dataset with a few numbers on location (mean, median), spread (variance, IQR) and shape (skew), turning raw columns into a mental picture.
The bell curve is the default error model of ML. Its 68-95-99.7 rule, z-scores and tail probabilities power outlier detection, confidence intervals, and almost every parametric test.
Hypothesis testing formalizes 'is this difference real?': null hypothesis, test statistic, p-value, Type I/II errors, and choosing the right test for the data.
A data science environment is Python plus numpy, pandas, matplotlib/scikit-learn and a project layout — managed with venvs or conda so notebooks and pipelines run identically anywhere.
Lists are ordered mutable sequences and dicts are O(1) key→value maps — the two structures nearly every data pipeline is built from; understanding their costs and semantics is core Python literacy.
List/dict/set comprehensions build collections in one expression, and zip, enumerate, sorted, map and lambda compose the functional-clean style pandas users write every day.
Manufacturing arrays — zeros, ones, arange, linspace, random fills, and reshaping between shapes — is the everyday grammar of feature matrices and images.
Ufuncs and broadcasting apply operations across whole arrays — including differently-shaped but compatible ones — making the classic loops-and-if cascade collapse into one line.
Fancy indexing, slices and boolean masks select exactly the rows, columns and elements you want — and choosing view vs copy determines whether you silently mutate your data.
np.random generates reproducible random data and np.linalg powers solve, eig, svd and matrix products — the primitives behind simulations, feature synthesis, PCA and least squares.
CSV, Parquet, Excel, JSON and SQL are the data interchange formats: reading, type inference, dtype control and fast binary formats are where real pipelines start.
.loc and .iloc are the two selection dialects — label-based and position-based — plus boolean masks and .at/.iat for scalars; the 'chained assignment' pitfall lurks for the unwary.
Tidy data, dtype fixing, missing values, duplicates and inconsistent strings: cleaning is 60% of real analysis, and pandas gives systematic tools for each.
split-apply-combine turns 'group by region, summarize revenue' into one pipeline: group keys, apply aggregations or transforms, and recombine — the pivot of all tabular analytics.
Matplotlib's figure/axes model, object-oriented API, styling, subplots and savefig are the porcelain beneath every Python chart, including seaborn's.
Seaborn turns dataframes into statistical graphics — relplots, displots, boxplots, heatmaps, regression fits — adding uncertainty and grouping with one call.
Plotly turns dataframes into hoverable, zoomable web charts with the same API breadth as matplotlib — the fastest path from notebook to shareable dashboard.
Exploratory Data Analysis is the deliberate, hypothesis-forming stage: shape, dtypes, nulls, distributions and relationships get examined BEFORE modeling so bad assumptions never reach the model.
One variable at a time: histograms, KDEs, value counts and summary stats characterize each feature's distribution, scale and sparsity before any relationship is examined.
Pairing variables reveals relationships: scatter/hexbins for two numerics, box/violin for numeric-by-category, correlation and crosstabs for categorical pairs — the raw material of feature ideas.
Missingness has a mechanism (MCAR/MAR/MNAR) and outliers have a cause (bug or signal) — analyzing both before deciding fixes separates thoughtful pipelines from delete-everything reflexes.
Accuracy hides class-imbalance failures. Precision, recall, F1 and the confusion matrix reveal exactly which errors a classifier makes and let you weight them by their real cost.
ROC and PR curves show how a classifier behaves at every decision threshold. AUC summarizes rank quality but hides calibration; PR-AUC is the honest surface for rare-event problems.
A score of 0.7 only means '70% likely' if probabilities are calibrated. Reliability curves, Brier score and Platt/isotonic rescaling keep decision systems from over-trusting softmax or sigmoid outputs.
A classifier's business value lives at a single operating point on its score curve. Choosing the threshold from cost matrices, expected profit, or constraints turns a raw model into a decision system.
K-fold is the workhorse CV strategy: k train/validate re-uses of the data with full coverage. Getting the fold count, seeds and error handling right is what makes it trustworthy.
Tuning inside plain CV leaks selection optimism into your estimate. Nested CV keeps model selection inside an inner loop and outer folds measure the honest performance of the whole tuning procedure.
Time ordered data can't be shuffled: the future must never seed the past. TimeSeriesSplit, expanding windows and blocked splits evaluate forecasting honestly while still 'reusing' data for tuning.
Grid search evaluates every combination of a discrete hyperparameter grid with cross validation. It's exhaustive, honest and embarrassingly parallel — but vulnerable to the dimensionality curse as the grid grows.
Random search samples hyperparameter combinations from distributions instead of an exhaustive lattice. With a fixed budget it covers the space better than an equally-sized grid, especially when only a few knobs matter.
Bayesian optimization builds a surrogate model of the score surface and picks the next config to maximize information per trial. It squeezes the most from expensive evaluations—ideal when each fit costs minutes or hours.
Early stopping halts training when validation stops improving; pruning cuts entire search candidates that can't win. Together they turn tuning budgets from 'number of full runs' into 'total compute spent', which is what actually matters.
Decomposition separates a series into trend, seasonal and residual parts. It's the diagnostic every forecaster runs first — and the simplest reliable forecasting tool when you model the components properly.
ARIMA models a series as autoregression plus lagged errors, made stationary by differencing. Knowing how order choices (p, d, q), ACF/PACF heuristics and fitted diagnostics work turns a black-box forecast into a defensible one.
SARIMA adds seasonal AR/MA/differencing terms to capture periodic patterns; ETS (error-trend-seasonality) models the components with smoothing instead. Choosing between them is a forecast-architecture decision.
Forecast errors must be measured against the right baseline and the right scale. RMSE, MAE, MASE, coverage and per-horizon profiles tell you whether a forecast is actually good — MASE and coverage especially when naive baselines are hard to beat.
Raw text is messy. Tokenization, normalization, stopword removal, stem/lemmatization and vocabulary handling turn strings into the clean token sequences every NLP model consumes.
Tickets, emails, reviews and support chats all become classifiers. From TF-IDF + linear models to embeddings to transformers, text classification pipelines share evaluation discipline that decides who wins.
Topic models discover latent themes across a corpus: LDA assigns documents to topic mixtures and topics to word distributions. They reveal structure in unlabeled collections and power exploratory NLP.
Semantic search compares meanings rather than keywords: embed queries and documents, then find nearest neighbors. Cosine similarity plus vector indexes and hybrid ranking make it production-ready.
Backpropagation computes the gradient of the loss with respect to every weight by applying the chain rule backward through the network, enabling gradient descent to actually train deep nets.
The loss scores every prediction and the optimizer decides how to descend it. Choosing them well is the difference between a loss curve that falls smoothly and one that oscillates, plateaus or diverges.
How weights start and how intermediate values are scaled determines whether deep networks learn at all. Xavier/Kaiming init and batch/group normalization keep activations in a stable regime.
PyTorch is the de-facto framework for research and production deep learning: automatic differentiation, GPU acceleration and a Pythonic imperative loop. This concept covers tensors, autograd, modules and a minimal training loop.
The perceptron is a single linear decision unit — a weighted sum plus a threshold. It separates linearly separable data perfectly but fails on anything else, which is exactly why multi-layer networks were invented.
Forward propagation pushes inputs through the layers, producing predictions and cached intermediates. It's the pass that computes outputs — and everything backprop needs.
Building a full MLP in numpy — init, forward, backward, optimizer, batching — turns framework magic into a checklist you can debug. This concept walks a complete training loop you can run in a sandbox.
Networks fail in recognizable patterns: NaN losses, plateaued plateaus, silent overfitting, degenerate predictions. This concept is your symptom → cause → fix playbook.
Convolution slides a small kernel over an image to detect patterns, while pooling downsamples the map. Padding, stride and kernel size govern the output geometry — and most shape bugs live here.
Transfer learning reuses a pretrained model's learned features for a new task: freeze the spine, replace the head, fine-tune. It converts 'need a million images' into 'need a few thousand'.
Object detection locates every instance of known classes inside an image — boxes plus labels. Two families dominate: two-stage (R-CNN) and one-stage (SSD/YOLO), trading accuracy for latency.
Serving CNNs adds a production-layer of concerns: export/quantization, batch inference, latency budgets, input pipelines, and how accuracy holds up at real-world resolutions and edge hardware.
An RNN feeds each input together with the previous hidden state through a weight-shared cell, carrying information across steps. Unrolled over time it's a deep shared-weight network — with all the gradient implications.
Seq2seq learns an input sequence → output sequence mapping: encoder reads one sequence into state, decoder generates the other. That framework powers translation, summarization and more — and attention fixed its context bottleneck.
RNNs solve forecasting, anomaly detection, speech-to-text, language models and more — wherever order matters. Modern stacks often swap RNN blocks for attention, but the sequence-applications playbook survives.
The gated math, formally: four gates (forget, input, candidate, output) combine linear maps over [h, x] with sigmoid/tanh activations. This concept nails the equations and shapes.
Time-series LSTM forecasting frames the series as a supervised window problem: build lagged windows, train a sequence-to-value model, and evaluate multi-step by re-injection. Baselines first; the LSTM earns its keep only when lags+classical models fall short.
Transformers replaced recurrent encoders for most NLP, but LSTM-style sequential modeling still appears beneath and beside them: in hybrid models, on-device decoders, and as the token-level recurrence of some long-context systems.
Self-attention lets tokens attend to other tokens in the SAME sequence, and multi-head runs several attention maps in parallel so different subspaces learn different relationships.
Self-attention is order-agnostic — shuffle the input and scores are identical. Positional encodings inject order as (learned or sinusoidal) vectors added to token embeddings so the model can know position.
Fine-tuning takes a pretrained transformer and adapts it with task data — full fine-tune, parameter-efficient (LoRA), or frozen-feature heads. The choice tunes the compute/quality trade-off.
Quadratic attention, KV-cache growth and 4x FFNs make transformers expensive. Efficiency stacks at every level: fused attention kernels, KV caching, quantization, speculative decoding and memory planning.
Greedy decoding always picks the argmax token; sampling with temperature, top-k and top-p trades determinism for diversity. Beam search and length penalties tune long-form quality. Decoding choices are product decisions.
RAG grounds LLM answers in retrieved documents: embed your knowledge base, fetch the most relevant chunks for a query, and feed them to the model with the question. It fixes hallucinations and staleness without retraining.
Shipping RAG means measuring retrieval and groundedness, controlling latency (caching, re-rankers, hybrid search), handling chunk updates, and building evaluation sets that catch regressions before users do.
Feature importance ranks how much each feature drives predictions: coefficients for linear models, impurity-based or permutation importance for trees, and drop-column tests for anything. Permutation importance is model-agnostic and honest about collinearity.
SHAP decomposes each prediction into additive per-feature Shapley values — how much each feature pushed the prediction away from the baseline. Consistent, locally accurate, and the default for explaining tabular models.
PDPs show how a feature affects predictions on average by sweeping its value and averaging the model output; ICE plots trace each individual unit. Together they reveal shape (linear? U-curve?) and heterogeneity the model learned.
Experiment tracking records every run: hyperparameters, metrics, artifacts and code version. Without it, 'which config beat the baseline?' is unknowable and every improvement is unverifiable. Tools: MLflow, W&B, custom SQLite + git.
Ablation isolates each component's contribution by removing it and measuring the pain. Removing a module, a feature group, or a trick from the full model shows which parts actually earn their keep.
Error analysis digs into WHERE a model fails: which classes, segments or slices concentrate the mistakes? Confusion matrices, per-slice metrics and clustering failures turn a bad score into a prioritized punch list.
A model artifact must round-trip: save weights + schema + metadata in one auditable bundle, stored in a registry where 'the model' means a versioned, promoted artifact — not a pickle in a repo.
Making inference fast and cheap without wrecking quality: quantization, batching/kernel fusion, smaller models (distillation), pruning, and caching. You optimize the cost/latency/quality triangle with an eval harness measuring all three.
A model API covers more than predict: request/response contracts, input validation, error handling, timeouts, observability and version pins. REST is the default; gRPC gives low-latency typed streaming.
Containers make a model deployable anywhere: code, deps, artifact and runtime in one reproducible image. Key arts: correct base image, small layers, GPU runtime support, health checks and artifact injection instead of baking.
Training infrastructure turns ad-hoc compute into scheduled, reproducible, shareable runs: GPU allocation, containers, orchestration, experiment parallelism and cost control — all pinned to the exact environment a model needs.
Data versioning, feature stores and train/serve consistency keep a model's inputs trustworthy: the same feature computed in training and in production, immutable dataset snapshots, and a 'feature as a service' contract.
CI/CD for ML automates the path from commit to registered model: linting and data tests on pull, training/evaluation gates on merge, and a deploy that is a registry promotion — plus continuous training for the data that keeps changing.
Monitoring models means watching the signals that predict decay: input distributions, prediction distributions, latency and proxies for quality — because true labels arrive late. Alerts, dashboards and review cadence make decay visible before users feel it.
Data drift is the input distribution moving away from training; concept drift is the mapping from inputs to labels changing. Both break models silently. Detection ranges from simple tests to adaptive systems, but the operational question is always 'what do we do when drift is real?'
Offline evaluation can't see distribution shift. Shadow deployments run a candidate on real traffic without deciding, canaries flip a small slice, and A/B tests measure business impact with statistical rigor. Each is a real-traffic interrogation with its own risk profile.
Retraining is a decision, not a reflex: schedule-, drift- or event-triggered; with cold-start vs warm-start and a backtest against historical windows so a 'fresh' model doesn't repeat old or regress from current behavior.