Linear Algebra in numpy
The math hidden inside every model, made direct
Matrix products, dot products, inverses, eigenvalues and least squares underpin regressions, similarity scoring and embeddings. numpy.linalg turns these into reliable one-liners.
What you will learn
- Multiply matrices correctly with @ and dot
- Solve linear systems and least squares
- Use eigenvalues/SVD for decomposition problems
What is this?
numpy.linalg includes dot/matmul (@), inv, solve, lstsq, eig, svd and norm. Dot products score similarity between feature vectors; @ multiplies matrices; solve handles square systems; lstsq fits least-squares regressions without building matrix inverses by hand; eigenvalues/SVD power principal components and recommendations.
Why do you need it?
- Regressions and similarity scores are matrix problems the moment you vectorize.
- @ and dot are the operations behind neural embeddings, PCA and cohorts-by-vector.
- solve/lstsq are numerically stable where hand-rolled inverses are not.
- Analysts who read linear algebra write models instead of consuming them.
Mental model
A logistics network routing goods through hubs
Matrix multiplication
Each hub combines supplies with capacities into a downstream pattern — rows of one matrix feeding columns of another. Small hub networks are fast to sum by hand; big ones are exactly what numpy's @ does for you, stably and instantly.
How it works
- a @ b multiplies matrices and vectors with correct contract conventions ((m,k)@(k,n) → (m,n)).
- np.dot is the classic spelling; @ is the modern operator.
- np.linalg.solve(A, b) solves Ax = b without computing an inverse.
- np.linalg.lstsq(A, b, rcond=None) fits least-squares coefficients for regression.
- np.linalg.eig(A) returns eigenvalues and vectors; svd(A) gives U, s, Vt for decomposition.
- norm, det, and matrix_power round out the toolbox for stability checks.
From problem to linalg call.
Examples
Least-squares slope and intercept
import numpy as npx = np.linspace(0, 10, 50)y = 2.5 * x + 1 + np.random.default_rng(1).normal(0, 0.5, 50)A = np.column_stack([x, np.ones_like(x)])coefs, *_ = np.linalg.lstsq(A, y, rcond=None)print(coefs.round(3)) # ~ [2.5, 1]print("similarity:", np.dot(x, y) / (np.linalg.norm(x) * np.linalg.norm(y)))lstsq recovers the slope/intercept from a design matrix; a normalized dot product scores similarity between two vectors.
In production
- A churn model scores user-feature vectors against a known-churn prototype with dot/norm similarity.
- A recommender computes item embeddings via SVD to surface alternatives.
- A demand forecaster solves a normal-equations regression with lstsq for the coefficient table.
Best practices
- Prefer solve over inv(A) @ b — faster and more stable.
- Use lstsq for overdetermined fits; it reports residuals ready for quality checks.
- Check matrix conditioning (cond) before trusting an inverse-driven result.
- Keep design matrices centered for interpretable intercepts.
Common mistakes
- Multiplying incompatible shapes with @ and blaming numpy for your transpose omission.
- Inverting a near-singular matrix and getting huge noise instead of solve.
- Using dot where you meant column-wise norms and watching broadcasting bite.
- Forgetting that lstsq needs an intercept column built explicitly.
Debugging tips
- A 'shapes (3,4) and (3,4) not aligned' error means a transpose is needed.
- Huge coefficient values → poor conditioning; center/scale the design columns.
- residuals big after lstsq → feature selection or a nonlinear pattern you're forcing linear.
- When eigenvalues explode, suspect scaling — normalize columns first.
Interview checkpoint
Quick quiz
The matrix product operator in modern NumPy is:
np.linalg.solve computes the matrix inverse before solving.