Multilayer Perceptron
Layers of neurons where every neuron connects to every neuron in the next layer. The simplest deep network, and a building block inside Transformers.
Also MLP · feedforward neural network · fully connected network · dense network
Rumelhart, Hinton and Williams, 1986
One judge can only draw one straight line. Put several judges in a row, each listening to the ones before, and together they can draw any shape you like.
Where it fits
Not shown: 2 more that build on it. See the full list below.
+All relationships as a list
Is a kind of / part of
Siblings
Uses / built on
Trained with
Evolved from
How it works
Take several perceptrons, arrange them in layers, and feed each layer’s outputs into the next. Replace the hard yes/no threshold with a smooth activation function so the whole thing is differentiable. That is a multilayer perceptron.
- Input layer: the features.
- Hidden layers: each neuron sees every output of the previous layer. Each layer re-describes the data in a new, more useful way.
- Output layer: the prediction, such as class probabilities.
Why does stacking help? Each hidden neuron draws one line; the next layer combines lines into regions; further layers combine regions into arbitrarily complex shapes. That solves XOR and, in principle, any pattern.
Training became practical with backpropagation, popularized in 1986.
The math
For a network with one hidden layer:
Without the nonlinearity this collapses to a single linear map , no more powerful than one layer.
Everyday use cases
- Case 01
Tabular predictions
Small MLPs predict from structured data, sometimes as part of a larger recommender system.
- Case 02
Inside every Transformer
Each Transformer block contains an MLP (the "feed-forward" layer) that processes every token.
Strengths, weaknesses and when to use it
Strengths
- Can model any continuous function given enough neurons.
- Simple and general.
Weaknesses
- Ignores structure in the input (neighboring pixels, word order), which CNNs and Transformers exploit.
- Parameter count grows quickly with input size.
When to use itFor small fixed-size inputs, or as a component inside a larger architecture.
Compare with
Check yourself
- Q01
What does an MLP add to the perceptron that lets it learn XOR?
Show answer
A: Hidden layers with nonlinear activations