Dimensionality Reduction
Methods that describe data with far fewer numbers while keeping what matters, to visualize it, speed up other models or remove noise.
Also feature extraction · projection · manifold learning
A shadow of a 3D object on a wall is a 2D summary of it. Pick the right angle for the light and the shadow still tells you what the object is.
Where it fits
+All relationships as a list
Is a kind of / part of
Kinds and parts
How it works
Real data often has many more columns than it has real “degrees of freedom”: a 64×64 face photo has 4,096 pixels but varies along far fewer meaningful directions. Dimensionality reduction finds a compact description.
Members in this atlas:
- PCA: the classic linear method, keeping the directions of greatest spread.
- t-SNE and UMAP: nonlinear maps that keep neighbors close, the standard way to draw embeddings in 2D.
- Autoencoders do the same job with a neural network.
Two uses, often confused: compression for later models (PCA is good, distances stay meaningful) and visualization (t-SNE and UMAP are good, but distances between far-apart clusters mean little). Related: the curse of dimensionality explains why reducing dimensions helps at all.
Everyday use cases
- Case 01
Visualizing embeddings
Squashing thousands of word or image vectors into a 2D map where similar items cluster together.
- Case 02
Survey analysis
Compressing 100 survey questions into a handful of underlying traits.
Strengths, weaknesses and when to use it
Strengths
- Makes high-dimensional data visible and fights the curse of dimensionality.
- Can denoise and speed up later models.
Weaknesses
- Some information is always lost; nonlinear maps can create misleading pictures.
When to use itBefore plotting high-dimensional data, or when many correlated features slow down or confuse a model.