GPUs and AI Accelerators
Specialized chips that do thousands of multiplications in parallel. Deep learning took off when graphics cards turned out to be perfect for neural networks.
Also GPU · TPU · NPU · CUDA · accelerator
A normal processor is a few brilliant chefs; a GPU is thousands of kitchen helpers who can each only chop. For a mountain of vegetables, the helpers win easily.
Where it fits
+All relationships as a list
How it works
Neural networks spend almost all their time on one operation: multiplying large matrices. That work splits naturally into millions of independent multiply-and-add steps, which is exactly what graphics cards were built for (computing millions of pixels in parallel).
AlexNet (2012) was trained on two consumer GPUs, and that demonstration helped start the deep learning boom. Since then:
- GPUs gained dedicated matrix units and fast memory specifically for AI.
- TPUs (Google’s tensor processing units) and other custom chips were designed for neural networks from the start.
- NPUs in phones and laptops run smaller models locally, for privacy and speed.
A subtle point: for many workloads, especially large language models generating text, the limit is how fast weights can be moved from memory, not how fast the chip can multiply. That is why quantization, which shrinks the weights, speeds things up.
Everyday use cases
- Case 01
Training large models
Large language models are trained on thousands of accelerators working together for weeks.
- Case 02
AI features on your phone
Built-in neural processing units run photo enhancement and speech recognition locally.
Famous examples
Strengths, weaknesses and when to use it
Strengths
- Orders of magnitude faster than ordinary processors for neural networks.
Weaknesses
- Expensive and energy hungry at scale.
- Memory, not raw compute, is often the bottleneck.
When to use itFor training any non-trivial neural network, and for serving at scale.
Check yourself
- Q01
Why are GPUs good at deep learning?
Show answer
A: Neural networks are mostly huge matrix multiplications, which split into many parallel operations