Neural Network
A model made of layers of simple connected units (neurons). With enough layers and data it can learn almost any pattern, which is what deep learning means.
Also artificial neural network · ANN · neural net · deep learning
A huge team of tiny workers passing notes along a line. Each worker adds up the notes they get and decides how loudly to pass a message on. With practice the team learns to recognize a cat in a photo.
Where it fits
Not shown: 8 more kinds and parts, 14 more it builds on. See the full list below.
+All relationships as a list
Is a kind of / part of
Kinds and parts
Siblings
Uses / built on
Trained with
Techniques that apply here
- Adam and Other Optimizers
- Adversarial Examples
- Attention
- Backpropagation
- Batch and Layer Normalization
- Data Poisoning
- Dropout
- Early Stopping
- Explainability
- GPUs and AI Accelerators
- Knowledge Distillation
- Mechanistic Interpretability
- Membership Inference and Data Extraction
- Pruning
- Quantization
- Residual Connection
- Weight Initialization
Used by
Evolved from
How it works
A neural network is built from neurons. Each neuron takes some numbers in, multiplies each by a weight, adds them up with a bias, and passes the result through an activation function. That is it. The power comes from wiring many of them together.
Neurons are arranged in layers. The input layer receives the raw data (pixel values, for example), hidden layers transform it step by step, and the output layer gives the prediction. Each layer builds on the previous one, so early layers in an image network detect edges, middle layers combine edges into shapes, and later layers recognize whole objects.
Training follows the usual recipe: compare predictions to the truth with a loss function, compute how every weight affects the loss with backpropagation, and nudge all weights with gradient descent.
Neural networks were invented decades ago (the perceptron dates to 1958) but only took over around 2012, when big datasets and GPUs made deep networks practical. AlexNet was the turning point.
The way the neurons are wired is the architecture: multilayer perceptrons, CNNs, RNNs and Transformers are all neural networks.
Everyday use cases
- Case 01
Face unlock and photo search
Networks recognize faces and objects in your photos.
- Case 02
Voice assistants
Speech is turned into text by neural networks.
- Case 03
Chatbots
Large language models are very large neural networks.
Famous examples
- 2012AlexNet
The deep convolutional network that won the 2012 ImageNet challenge by a huge margin and kicked off the deep learning boom.
- 2020GPT-3
OpenAI's 175-billion-parameter language model that showed a big enough model can do new tasks from just a few examples in its prompt.
- 2023Claude
Anthropic's family of AI assistants: Transformer-based large language models trained with RLHF and Constitutional AI to be helpful, honest and harmless.
Strengths, weaknesses and when to use it
Strengths
- Learns features automatically from raw data (pixels, audio, text).
- Keeps improving with more data and compute.
Weaknesses
- Needs lots of data and compute.
- Hard to interpret why a decision was made.
When to use itFor unstructured data like images, audio and text, or when you have very large datasets.
Compare with
Check yourself
- Q01
What makes a neural network "deep"?
Show answer
A: Many layers
- Q02
Why are activation functions needed?
Show answer
A: Without them, stacked layers collapse into a single linear function