Mathematics
Beginner 14 min read Mathematics

Vectors and Dot Products

The atom of machine learning data

A vector is an ordered list of numbers; the dot product measures how aligned two vectors are and is the fundamental operation behind predictions and similarities alike.

vectordot productmagnitudenormcosine similarity
Prerequisites: Why Mathematics Matters
Before we start

What you will learn

  • Represent a feature row as a vector
  • Compute norms and dot products by hand and in numpy
  • Relate the dot product to angle and similarity between data points
Introduction

What is this?

A vector is an ordered list of numbers. In ML, every row of data — a customer, an image, a sentence — becomes a vector, and all geometry then applies: distance, direction, angle, similarity. The dot product of two same-length vectors sums the products of corresponding entries. Geometrically it measures how much two vectors point the same way, which makes it the engine of linear models and similarity search.

Motivation

Why do you need it?

  • A linear model's prediction is exactly a dot product: weights · features.
  • Cosine similarity — the normalized dot product — powers recommendation engines and search.
  • Distances (euclidean) between vectors drive k-nearest neighbors and clustering.
  • Mastering vectors makes neurons, embeddings and matrix multiplication feel like one idea.
Think about it

Mental model

Imagine

Checking a weather forecast vs a friend's outlook on the same 5 dimensions

Think of

Dot product measuring agreement

If both rate 'rain' high and 'wind' low, their dot product is large — they agree. Dot products are agreement scores between feature vectors.

Mechanics

How it works

  • A vector v[3] = [2, 4, 6] is a point/arrow in 3D space; its length (magnitude, L2 norm) is √(2² + 4² + 6²).
  • The dot product of u and v is Σ uᵢvᵢ — a single number.
  • Dividing the dot product by both magnitudes gives cosine of the angle θ between them, ranging from -1 (opposite) to +1 (identical direction).
  • A linear model ŷ = w·x + b is a dot product of weights and features plus a bias.
  • numpy computes all of this in one line: np.dot, np.linalg.norm, or the @ operator.

From feature row to geometric object.

Feature row
Encode as vector
Dot product
Cosine similarity
Decision
Hands-on

Examples

Dot product and similarity in numpy

Dot product and similarity in numpy
import numpy as np
u = np.array([3, 4])          # vector of norms, length 5
v = np.array([6, 8])          # parallel to u, length 10
w = np.array([-4, 3])         # perpendicular to u
 
print(u @ v)                  # 50  = |u||v|cos(0)
print(u @ w)                  # 0   = perpendicular, cos(90)=0
 
cos_sim = (u @ v) / (np.linalg.norm(u) * np.linalg.norm(v))
print(cos_sim)                # 1.0 — perfectly aligned

Parallel vectors give the maximum dot product relative to their size; perpendicular vectors give zero. Cosine similarity rescales the dot product so the magnitude of the vectors doesn't dominate the score.

Real world

In production

  • A recommendation engine ranks items by cosine similarity between a user's embedding vector and item embedding vectors — a batched dot product.
  • A search team encodes queries and documents into vectors and retrieves the top-k by dot product with a hardware-optimized matrix multiply.
  • Feature stores ship entire matrices to GPUs where one @ operation scores millions of records for a linear model.
Guidelines

Best practices

  • Always check vector lengths match before a dot product — shape errors are the most common algebra bug.
  • Normalize vectors before comparing when you care about direction, not magnitude.
  • Prefer the @ operator or np.dot over manual loops — they're vectorized.
Watch out

Common mistakes

  • Confusing element-wise multiplication (*) with the dot product (@) in numpy.
  • Using raw dot products for similarity when vector magnitudes vary wildly; normalize first.
  • Treating 'jaccard' or other set-based similarity as if it were cosine — different semantics.
When things break

Debugging tips

  • For shape mismatch, print both arrays' .shape before the operation.
  • If similarity scores cluster oddly, check for NaN introduced by zero-length vectors (divide by zero).
  • Verify dot products by hand on a 2D example once — if the code matches the hand calc, trust it.
Interviews

Interview checkpoint

Practice

Hands-on lab

Vectors, Norms and Cosine Similarity
25 min · beginner
In progress
Objectives
  • Build vectors from raw data rows
  • Compute norms and dot products with numpy
  • Compare euclidean distance vs cosine similarity on real values

Vectors, Norms and Cosine Similarity

25 min · beginner

0/5 steps
0%
sandbox · Linux sandbox
$
Try these commands
Test yourself

Quick quiz

0/2
1

u = [1,2,3], v = [4,5,6]. What is u·v?

2

Two perpendicular vectors have a dot product of zero.

Go deeper

Official documentation

numpy dot / matmul
docs
Essence of Linear Algebra — Vectors
video