Autoencoder
A network trained to copy its input through a narrow bottleneck, which forces it to learn a compact summary of the data.
Also AE · encoder-decoder · bottleneck network
Hinton and Salakhutdinov, 2006 (idea dates to the 1980s)
Describe a photo to a friend using only ten words, and they have to redraw it. To do well, you both have to agree on which ten words capture the most important things.
Where it fits
+All relationships as a list
Is a kind of / part of
Siblings
Uses / built on
Evolved into
How it works
An autoencoder has two halves:
- The encoder squeezes the input (say a 784-pixel image of a digit) down to a short list of numbers, the code or latent vector (say 16 numbers).
- The decoder expands those 16 numbers back into a 784-pixel image.
Training compares the output with the original and adjusts both halves to make the reconstruction as close as possible. No labels needed: the input is its own target.
Because the code is much smaller than the input, the network cannot memorize pixels. It must discover the few underlying factors that matter: stroke thickness, slant, which digit. That compressed code is useful for visualization, search and detecting unusual inputs.
As a generator, though, a plain autoencoder disappoints: pick a random code and the decoder usually produces a blur, because the codes it learned are scattered with empty space between them. The VAE fixes this.
Everyday use cases
- Case 01
Image compression and denoising
Train on noisy-in, clean-out pairs and the autoencoder learns to remove noise.
- Case 02
Anomaly detection
Something the model reconstructs badly is unlike the training data, for example a defect on a production line.
Strengths, weaknesses and when to use it
Strengths
- Learns compact representations without labels.
- Simple to train.
Weaknesses
- The code space has gaps, so random codes decode to garbage. Not a good generator on its own.
When to use itFor compression, denoising and anomaly detection. For generating new samples, use a VAE or diffusion model.
Compare with
Check yourself
- Q01
What is an autoencoder trained to output?
Show answer
B: Its own input
- Q02
Why is the bottleneck important?
Show answer
A: Without it the network could just copy the input without learning anything useful