Online Learning
Learning from data that arrives one example (or small batch) at a time, updating the model continuously instead of retraining from scratch.
Also incremental learning · streaming learning · continual learning · catastrophic forgetting
Instead of reading the whole textbook before the exam, you learn a little from every new lesson and adjust what you know as you go.
Where it fits
+All relationships as a list
How it works
Most models are trained in batch mode: collect a dataset, train, deploy, and retrain later. Online learning updates the model as each new example arrives.
It fits naturally with stochastic gradient descent, which already learns one small batch at a time. The model predicts, sees the true answer, and takes a small step.
Why use it:
- The world changes. Prices, fashions and spam tactics shift. See data drift.
- The data is too big to store or arrives faster than you could retrain.
The risks are the flip side: a burst of bad data can push the model off course (see data poisoning), and deep networks trained only on recent data tend to forget older skills, a problem called catastrophic forgetting. Production systems often combine both modes: frequent small updates plus periodic full retraining.
Everyday use cases
- Case 01
Ad click prediction
Models update on fresh clicks every few minutes because what people click on changes quickly.
- Case 02
Spam filtering
Spammers change tactics daily, so filters keep learning from new user reports.
Strengths, weaknesses and when to use it
Strengths
- Adapts quickly to change; needs little memory.
Weaknesses
- Sensitive to bad or adversarial data arriving in the stream.
- Neural networks can forget older patterns when updated only on new data.
When to use itWhen data arrives continuously and its patterns shift over time.
Compare with
Check yourself
- Q01
What is catastrophic forgetting?
Show answer
A: A model losing old skills when trained only on new data