ROC Curve and AUC
A curve showing a classifier's true positive rate against its false positive rate at every threshold; the area under it (AUC) summarizes ranking quality in one number.
Also ROC · AUC · ROC-AUC · receiver operating characteristic · area under the curve · true positive rate · false positive rate
A smoke alarm can be set to be very sensitive (catches every fire, but also burnt toast) or relaxed (no false alarms, but misses small fires). The ROC curve shows every possible setting at once.
Where it fits
+All relationships as a list
Is a kind of / part of
Siblings
- Accuracy
- Bias-Variance Trade-off
- Cross-Validation
- Curse of Dimensionality
- Data, Features and Labels
- Gradient Descent
- Hyperparameters
- Linear Algebra Basics
- Log Loss (Cross-Entropy)
- Loss Function
- Overfitting
- Precision, Recall and F1
- Probability Basics
- Regression Metrics
- Regularization
- Train, Validation and Test Sets
Uses / built on
How it works
Most classifiers output a score, and you choose a threshold above which you say “positive”. Each threshold gives a different confusion matrix, so which one should you report?
The ROC curve shows them all. For every threshold, it plots:
- True positive rate (recall): the share of real positives you catch.
- False positive rate: the share of real negatives you wrongly flag.
A strict threshold sits near the bottom-left (few alarms of either kind). A loose threshold sits near the top-right (catch everything, flag everything). A good model bends toward the top-left corner.
AUC is the area under the curve. It has a neat meaning: pick one random positive and one random negative example; AUC is the chance the model scores the positive one higher.
- 0.5: no better than a coin flip.
- 0.7 to 0.8: useful.
- above 0.9: strong (or suspiciously strong; check for leakage).
When positives are very rare, as in fraud, the false positive rate stays small even with many false alarms. Then precision matters more, and the precision-recall curve is the better view.
Everyday use cases
- Case 01
Medical screening
Choosing the operating point of a cancer screening model that balances missed cases against unnecessary follow-up scans.
- Case 02
Comparing fraud models
AUC compares how well two models rank risky transactions before anyone picks a threshold.
Strengths, weaknesses and when to use it
Strengths
- Independent of the chosen threshold and of class balance in the score.
- Has a clear meaning as a ranking probability.
Weaknesses
- On very imbalanced data a high AUC can hide poor precision; check the precision-recall curve too.
- Says nothing about calibration (whether 0.9 really means 90%).
When to use itTo compare binary classifiers by how well they rank, before choosing a threshold.
Compare with
Check yourself
- Q01
What AUC does random guessing give?
Show answer
B: 0.5
- Q02
Lowering the decision threshold usually...
Show answer
A: Raises both the true positive rate and the false positive rate