Federated Learning
Training one shared model across many devices or organizations without collecting their raw data: each trains locally and only sends model updates.
Also on-device learning · federated averaging · FedAvg
McMahan et al., Google, 2016
Instead of everyone mailing their diaries to one person to learn from, each person studies their own diary and only sends back a short note of what they learned.
Where it fits
+All relationships as a list
How it works
Usually, training data is gathered on one server. Federated learning reverses that: the model travels to the data.
- A server sends the current model to many devices (or organizations).
- Each trains it briefly on its own local data with gradient descent.
- Each sends back only its model update.
- The server averages the updates into a better shared model, and the cycle repeats.
Google introduced the approach in 2016 and used it to improve phone keyboard suggestions without uploading what people type. It is also used by groups of hospitals and banks that cannot share records.
Not sending data is not the same as perfect privacy: updates can reveal details about the data behind them. Serious deployments add secure aggregation (the server only sees the sum of updates) and differential privacy.
Everyday use cases
- Case 01
Phone keyboard predictions
Next-word models have been improved from typing on phones without the text leaving the device.
- Case 02
Hospitals collaborating
Several hospitals train a shared model without sharing patient records.
Famous examples
Strengths, weaknesses and when to use it
Strengths
- Raw data stays where it is, which helps privacy and regulation.
Weaknesses
- Updates can still leak information unless protected.
- Devices differ in data, speed and availability, which complicates training.
When to use itWhen data cannot be centralized for privacy, legal or bandwidth reasons.
Compare with
Check yourself
- Q01
What leaves a device in federated learning?
Show answer
B: Model updates