Edge AI (On-Device ML)
Running models directly on phones, cars, cameras and sensors instead of in the cloud, for speed, privacy and offline use, which forces models to be small and efficient.
Also on-device AI · TinyML · NPU · mobile ML · embedded ML
Instead of phoning a faraway expert every time, the device carries a pocket-sized expert of its own. It answers instantly, works without internet, and your data never leaves home.
Where it fits
+All relationships as a list
Is a kind of / part of
Siblings
Uses / built on
How it works
Most big AI models run in data centers (model serving). Edge AI runs them on the device itself: your phone, your car, a doorbell camera, a factory sensor.
Why bother?
- Speed: no network round trip. A car’s emergency braking cannot wait for the cloud.
- Privacy: data stays on the device. Face ID never sends your face anywhere.
- Offline: works in a tunnel or on a plane.
- Cost: the device’s chip does the work, not your servers.
The price is tight limits on memory, compute, battery and heat. Edge models are made small with:
- Quantization: fewer bits per weight.
- Pruning: removing unneeded weights.
- Distillation: training a small student to imitate a big teacher.
- Efficient architectures designed for mobile chips, and dedicated neural processing units (NPUs) in modern phones and laptops (hardware accelerators).
Edge devices can even learn together without sharing raw data, which is federated learning.
Everyday use cases
- Case 01
Face unlock
Face ID's matching runs entirely on the phone, so the face template never leaves it.
- Case 02
Wake words and live captions
Phones detect "Hey Siri" and caption videos without sending audio to a server.
Famous examples
- 2017Face ID and Face Recognition
Unlocking a phone with your face: a depth camera and neural networks compare your face to a stored template, the best-known everyday use of face recognition.
- 2018Keyboard Autocomplete
The word suggestions above your phone keyboard: small on-device language models that predict your next word, in Gboard's case trained with federated learning so your typing stays on your phone.
Strengths, weaknesses and when to use it
Strengths
- Instant response, works offline, keeps personal data on the device.
Weaknesses
- Tight memory and power budgets limit model size and quality.
- Updating and monitoring models on millions of devices is harder.
When to use itWhen latency, privacy, connectivity or serving cost rules out the cloud.
Compare with
Check yourself
- Q01
Which technique is commonly used to fit a model onto a phone?
Show answer
A: Quantization