Intermediate 14 min read Clustering

KMeans Clustering

Centroids and assignments, looped to convergence

KMeans assigns every row to the nearest of K centroids and iterates until stable — a fast, scalable, convex-ish solver for round-ish clusters, blind to shapes and outlier-heavy data.

KMeanscentroidsinertiaLloyd's algorithmk-means++initialization

Prerequisites: Clustering Basics

Before we start

What you will learn

Walk through Lloyd's iteration: assign -> update -> converge
Explain inertia and elbow, and the k-means++ initialization rationale
Know KMeans' failure modes: non-spherical, uneven size, outliers

Introduction

What is this?

KMeans partitions n rows into K clusters each represented by a centroid (mean). Algorithm (Lloyd): initialize K centroids (k-means++ spreads them), assign each row to the nearest centroid, recompute centroids as cluster means, repeat until assignments stop changing. It minimizes within-cluster inertia (WSS) locally. Fast, O(n*K*d) per iteration, scales easily — but assumes roughly spherical, evenly sized clusters.

Motivation

Why do you need it?

It's the default segmenter: milliseconds, interpretable centroids, ubiquitous library support.
The assignment geometry carries over to vector quantization, embeddings and retrieval.
Its failure modes teach the difference between 'clusters' and 'blobs'.
Elbow/silhouette methodology (next concept) is built on its inertia/silhouette outputs.

Think about it

Mental model

Imagine

Towns rallying around the nearest mayor

Think of

KMeans assignment and recenter

People (rows) move to the nearest town (centroid); the mayor re-centers the town on its population; boundaries re-trace. When nobody moves again, the map is settled — usually a good partition, occasionally an artificial line through a valley.

Mechanics

How it works

Choose K; initialize centroids with k-means++ (spread-out starts, avoids dead clusters).
Assign: each row nearest centroid by Euclidean distance (Voronoi cells).
Update: recalculate each centroid as the mean of its assigned rows (minimizes WSS locally).
Converge when assignments stable (typical <100 iters); run several restarts and keep best WSS.
Tune K by elbow/silhouette; interpret centroids as representative profiles.

Lloyd's two-step loop.

Initk-means++ centroids

Assignnearest centroid

Updatemean of assignments

Stableassignments fixed

Hands-on

Examples

KMeans on blobs with centroid profile

import numpy as np
import pandas as pd
from sklearn.datasets import make_blobs
from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler
 
X, y = make_blobs(n_samples=900, centers=4, n_features=3, random_state=0)
Xs = StandardScaler().fit_transform(X)
 
km = KMeans(n_clusters=4, n_init=20, random_state=0).fit(Xs)
print("inertia (WSS):", round(km.inertia_, 3))
 
profiles = pd.DataFrame(Xs, columns=["f1", "f2", "f3"])
profiles["cluster"] = km.labels_
print(profiles.groupby("cluster").mean().round(3))
print("centroids match group means:", np.allclose(km.cluster_centers_, profiles.groupby("cluster").mean().values, atol=1e-9))

After convergence, cluster means equal the fitted centroids by construction — that identity is the whole algorithm.

Real world

In production

Segmentation across spend channels: centroids become brand personas for campaigns.
Vector quantization: product embeddings quantized to K prototype vectors for KNN speedup.
Cold-start clusters: assign new users to nearest centroid -- instant targeting.

Guidelines

Best practices

Standardize features so centroid meaning is unit-free; set n_init >= 10-20; try several K and validate with stability; inspect cluster sizes and drop tiny singleton clusters before calling them 'segments'.

Watch out

Common mistakes

Forcing K on elongated/diagonal clouds (Blob bias); ignoring outliers that become orphan centroids; trusting one n_init=1 result; treating inertia alone as 'best K'.

When things break

Debugging tips

Empty cluster after fit: your K exceeded the data's structure — trim K.
Centroids colliding (near-duplicate): reduce K or use better space.
Squished clusters in a 2D project: often an unscaled-feature artifact.

Interviews

Interview checkpoint

Test yourself

Quick quiz

0/2

Each KMeans iteration does:

A large cluster with a few extreme rows usually breaks KMeans because:

Go deeper

Official documentation

KMeans (sklearn)

docs

k-means clustering | Wikipedia