Data Drift
When the data a model sees in production gradually differs from its training data, so its accuracy quietly degrades.
Also concept drift · distribution shift · model monitoring · covariate shift
You learned to drive in summer. Then winter comes with snow and ice. You did not get worse at driving; the roads changed.
Where it fits
+All relationships as a list
How it works
Models assume the future looks like the past they were trained on. Production regularly breaks that assumption:
- Covariate shift: inputs change. A new phone camera makes photos brighter; a marketing campaign brings in younger customers.
- Concept drift: the meaning changes. What counts as spam evolves; fraud tactics adapt.
Unlike a crash, drift is silent: the model still answers confidently, just more often wrongly. So deployed models need monitoring:
- Track input statistics (averages, missing values, category frequencies) and compare them with training data.
- Track the distribution of predictions (a fraud model suddenly flagging twice as many transactions is a red flag).
- When true labels arrive, track real accuracy over time.
The usual remedy is retraining on recent data, on a schedule or when monitoring raises an alarm. Drift is also why a strong test score at launch, while necessary (see overfitting), is never the end of the story.
Everyday use cases
- Case 01
Shopping during a crisis
Sudden changes in behavior, such as panic buying, broke many demand-forecasting models overnight.
- Case 02
Fraud patterns
Fraudsters adapt to detection, so yesterday's patterns stop working.
Strengths, weaknesses and when to use it
Strengths
- Monitoring for it catches problems before users or revenue do.
Weaknesses
- True accuracy often cannot be measured live because labels arrive late.
When to use itFor every deployed model: decide what to monitor before launch.
Check yourself
- Q01
A spam filter gets worse because spammers change their wording. What is this?
Show answer
A: Concept drift