Differential Privacy
A mathematical guarantee that a model or statistic reveals almost nothing about any single person in the data, achieved by adding carefully calibrated noise.
Also DP · DP-SGD · privacy budget · epsilon
Dwork, McSherry, Nissim and Smith, 2006
If removing your diary from a big pile of diaries would not change the results noticeably, then the results cannot reveal your secrets.
Where it fits
+All relationships as a list
Is a kind of / part of
Siblings
Applies to
Used by
How it works
Models can leak their training data. A membership inference attack, for example, guesses whether a specific person’s record was in the training set from how confidently the model handles it. Simply removing names does not stop this.
Differential privacy offers a precise promise: whatever the model outputs would be almost equally likely whether or not any single person’s data was included. If your data barely changes the result, the result cannot reveal much about you.
The usual recipe for training models is DP-SGD, a modified gradient descent:
- Compute each example’s gradient separately and clip it, so no single person can have a large influence.
- Add random noise to the combined gradient.
- Track the total privacy “budget” ε spent over all training steps.
Smaller ε means stronger privacy but more noise and lower accuracy. The trade-off is real, and choosing ε involves policy, not just engineering. It is often combined with federated learning.
Everyday use cases
- Case 01
Census statistics
The 2020 US Census used differential privacy to protect individuals in published tables.
- Case 02
Learning from phones
Combined with federated learning to learn from typing or usage patterns.
Famous examples
Strengths, weaknesses and when to use it
Strengths
- A provable guarantee that holds against any attacker, whatever they already know.
Weaknesses
- The noise costs accuracy, especially for small datasets or rare groups.
- Choosing the privacy budget ε is partly a policy decision.
When to use itWhen models or statistics are built from sensitive personal data and must resist attacks such as membership inference.
Compare with
Check yourself
- Q01
What does a smaller ε mean?
Show answer
A: Stronger privacy and more noise