Unsupervised Learning
Intermediate 12 min read Unsupervised Learning

t-SNE Visualization

See local structure, distrust the pixels

t-SNE embeds high-dimensional points into 2-3D for visualization by preserving local neighborhoods — beautiful for exploring clusters, unsafe for drawing solid conclusions.

t-SNEembeddingvisualizationperplexitymanifoldlocal structure
Prerequisites: PCA and Dimensionality Reduction
Before we start

What you will learn

  • Explain what t-SNE preserves (local neighbors) and what it distorts (distances, density, global shape)
  • Use perplexity and early exaggeration sensibly
  • Interpret t-SNE plots carefully — they are views, not measurements
Introduction

What is this?

t-SNE builds a low-dimensional map (usually 2D) where similar high-dimensional points sit close and dissimilar ones sit apart. It models pairwise similarities as probabilities, then optimizes (KL divergence) so the low-D distribution matches the high-D one. Key knobs: perplexity (roughly the local neighborhood size ~5-50), early exaggeration (cluster separation during warm-up), and n_iter.

Motivation

Why do you need it?

  • Reduction + visualization turns '40-dimensional mess' into a scatter plot a stakeholder can look at.
  • Local preservation often reveals clusters hammering separations PCA smooths away.
  • Perplexity sweeps help you sanity-check whether structure is stable or an artifact.
  • It's the everyday tool for spot-checking embeddings (LLM vectors, image features).
Think about it

Mental model

Imagine

A fold-out map of a city's neighborhoods

Think of

t-SNE's local-structure view

The map shows which neighborhoods are adjacent and how streets connect locally; it doesn't faithfully encode 'land size' or the true distances between distant areas. Great for wandering, wrong for measuring.

Mechanics

How it works

  • Compute pairwise similarity of every row (usually on PCA-reduced features for speed).
  • Model high-D neighbor probabilities with per-point bandwidths chosen by perplexity.
  • Randomly initiate low-D points; gradient-walk distances to match the probability distribution.
  • Early exaggeration widens early cluster separation, then cools to exact matching.
  • Result: coordinates that recompute 'which rows are nearest in the original space' as proximity.

t-SNE pipeline.

Reduce
Similarities
Optimize
Plot
Hands-on

Examples

Digits embedded with two perplexities

Digits embedded with two perplexities
import numpy as np
from sklearn.datasets import load_digits
from sklearn.decomposition import PCA
from sklearn.manifold import TSNE
 
digits = load_digits()
X = PCA(n_components=30, random_state=0).fit_transform(digits.data)
y = digits.target
 
for perp in [5, 30, 50]:
    emb = TSNE(n_components=2, perplexity=perp, random_state=0).fit_transform(X)
    by_class = [emb[y == k].mean(axis=0) for k in range(10)]
    sep = np.std([np.linalg.norm(c - np.mean(by_class, axis=0)) for c in by_class])
    print(f"perplexity {perp:>2}: class-centroid spread {sep:.2f}")

Changing perplexity shifts how much local vs global structure dominates — the class-centroid spread moves, a reminder that t-SNE reads are view-dependent.

Real world

In production

  • Embedding QA: engineers eyeball LLM vector clusters for weird mixtures before serving.
  • Data quality patrol: t-SNE of unlabeled traffic exposes buttoned-off populations pre-labeling.
  • Segments presentation: a colored t-SNE scatter communicates cluster existence to executives simply.
Guidelines

Best practices

  • Run PCA (50-100 dims) before t-SNE for speed and denoising; sweep perplexity (5/15/30/50) and report that clusters persist; fix the same random seed and row order for reproducible plots; label your plot axes as 't-SNE-1/2', NOT as features.
Watch out

Common mistakes

  • Reading distances or cluster sizes from t-SNE (density and global size are distorted); trusting a single perplexity; using t-SNE coordinates as model features; claiming real well-separation from one run.
When things break

Debugging tips

  • If clusters vanish as perplexity changes, they're probably not robust.
  • Beware 'string of pearls' artifacts from pipelines trivial rows (they're an embedding artifact, not a structure).
  • Re-run with 2 seeds: wildly shifted layout = fine (embeddings are stochastic), but cluster MEMBERSHIP should stay mostly stable.
Interviews

Interview checkpoint

Test yourself

Quick quiz

0/2
1

t-SNE preserves primarily:

2

Best use of a t-SNE plot:

Go deeper

Official documentation

T-SNE (sklearn manifold)
docs
How to Use t-SNE Effectively (Distill)
video
t-distributed stochastic neighbor embedding | Wikipedia
docs