Cluster Evaluation
Check clusters from three angles
No labels means validate by geometry (silhouette, inertia), by agreement (ARI/NMI against a reference, cluster stability), and by usefulness (do segments move a business metric?).
What you will learn
- Use silhouette/inertia for internal quality
- Use ARI/NMI for agreement with a reference or bootstrapped re-clustering
- Close every cluster review with a usefulness test
What is this?
Cluster evaluation has internal, external and practical flavors. Internal: silhouette (a in own vs nearest neighbor cluster), inertia (compactness), Davies-Bouldin. External: ARI (label match vs a reference — also bootstrap stability), NMI. Practical: do clusters change a downstream business metric? A cluster 'exists' when all three pass.
Why do you need it?
- Silhouette alone misleads when data lacks real clusters — the numbers still produce one.
- Stability catches seed/subsample-dependent artifacts that silhouette can't.
- External agreement (ARI) is the honest way to compare clusterings.
- Usefulness is the metric of record: 5 fake segments vs 4 usable ones.
Mental model
Reviewing a restaurant: crowd vibe (geometry), repeat-customers (stability), actually good food (usefulness)
Evaluating clusters
A full house is nice but proves little; recurring faces say the draw is real; meals people reorder decide the review. Same for clusters: score them on all three, not one.
How it works
- Silhouette: for each row s(i) = (b-a)/max(a,b) with a within-cluster, b nearest-other; average all rows for quality [-1,1].
- Inertia/Davies-Bouldin: compactness measures; sweep k on the elbow/peak.
- ARI/NMI: compare assignment vs a reference or re-cluster on a subsample (stability agreement).
- Practical: fit a downstream decision; test whether segment means/prediction power differ from chance.
- Combine: real clusters = good silhouette AND stable AND useful.
Three-angle validation.
Examples
Silhouette and ARI sanity
import numpy as npfrom sklearn.datasets import make_blobsfrom sklearn.cluster import KMeansfrom sklearn.metrics import silhouette_score, adjusted_rand_score X, y = make_blobs(n_samples=700, centers=4, n_features=3, random_state=0)Xs = (X - X.mean(0)) / X.std(0) km = KMeans(n_clusters=4, n_init=20, random_state=0).fit(Xs)print("silhouette @4:", round(silhouette_score(Xs, km.labels_), 3)) # stability: re-run on half the data, compare agreementX2 = Xs[::2]km2 = KMeans(n_clusters=4, n_init=20, random_state=1).fit(X2)print("ARI (full vs half split):", round(adjusted_rand_score(km.labels_[::2], km2.labels_), 3))Near-textbook blobs give high silhouette and ARI ~1 between full and half-data fits — the stability bar clusters must pass.
In production
- Segmentation review: ARI across campaign cycles proves segments persist between refreshes.
- Model stepping: usefulness check — segment-separated churn rates actually differ before segments are declared.
- Audit trail: silhouette + stability figures go in the deck that authorizes the market roll-out.
Best practices
- Report mean AND fold-spread of silhouette; include a no-clusters baseline (one cluster) in the comparison; count 'durable vs cosmetic' clusters; always end with a usefulness question the segmenter can answer.
Common mistakes
- Trusting a single high silhouette; confusion of 'stable across seeds' with 'true structure'; ignoring the 1-cluster comparison (best silhouette sometimes = 1 cluster); dropping usefulness because the numbers look good.
Debugging tips
- Silhouette ~0 across k: no k produces coherent clusters — say so, don't force it.
- ARI low between seeds AND full/half: treat segments as weak despite elbow cosmetics.
- If a business split (sales region) out-AGRs your clusters, your features don't carry the signal.
Interview checkpoint
Quick quiz
Silhouette near +1 means rows are:
ARI measures: