Anomaly Detection
Finding the rare data points that look unlike the rest, such as fraudulent payments, failing machines or intrusions, usually without labeled examples of the problem.
Also outlier detection · novelty detection
Liu, Ting and Zhou (isolation forest), 2008
In a flock of white sheep, spot the black one. You don't need to know what a black sheep is; you only need to know what normal looks like.
Where it fits
+All relationships as a list
Is a kind of / part of
Siblings
Uses / built on
Applies to
Techniques that apply here
Used by
How it works
Anomalies are rare and varied: tomorrow’s fraud may look nothing like yesterday’s. So instead of learning what problems look like, anomaly detection usually learns what normal looks like and flags whatever does not fit. That makes it mostly unsupervised.
Common approaches:
- Statistics: flag values far from the usual range for that customer, machine or hour.
- Distance and density: points far from all clusters, or in very sparse regions.
- Isolation forest: build random trees that split the data at random; anomalies end up alone after only a few splits, while normal points need many.
- One-class SVM: draw a tight kernel boundary around the normal data and flag points outside it.
- Reconstruction: train an autoencoder on normal data; inputs it reconstructs badly are suspicious.
The practical challenge is the threshold: flag too much and analysts drown in false alarms; flag too little and real problems slip through. Feedback from investigated cases is often used later to build a supervised model on top.
Everyday use cases
- Case 01
Card fraud
Flag a purchase that is unusual for this card in amount, place or time.
- Case 02
Predictive maintenance
Sensor readings drifting from normal warn of a machine about to fail.
- Case 03
Network security
Unusual login patterns or traffic can signal an intrusion.
Famous examples
Strengths, weaknesses and when to use it
Strengths
- Works without examples of every possible problem.
Weaknesses
- Unusual is not the same as bad: false alarms are common.
- What counts as normal changes over time.
When to use itWhen problems are rare, varied or unknown in advance, so you cannot collect labels for them.
Compare with
Check yourself
- Q01
Why does an isolation forest flag outliers quickly?
Show answer
A: Unusual points can be separated from the rest with very few random splits