Instance-Based Learning
Methods that keep the training examples themselves and predict by comparing a new input to the most similar stored ones, instead of learning a formula.
Also memory-based learning · lazy learning · nearest-neighbor methods · non-parametric methods
Instead of memorizing rules, you keep a photo album of every dog and cat you have seen. For a new animal, you flip to the most similar photos and go with what they show.
Where it fits
+All relationships as a list
Is a kind of / part of
Kinds and parts
How it works
Most models compress the training data into parameters: a line’s slope, a tree’s splits, a network’s weights. Instance-based methods skip that step. They keep the examples and decide at prediction time, by looking up the most similar ones.
The classic member is k-nearest neighbors: find the closest training points and let them vote. Everything depends on what “close” means, which is why feature scaling matters so much here and why the curse of dimensionality hurts.
The same idea powers modern search: comparing embeddings in a vector database is nearest-neighbor lookup at scale, just in a learned space instead of raw features.
Everyday use cases
- Case 01
"People who liked this also liked"
Recommending items by finding the customers most similar to you.
- Case 02
Similarity search
Finding the closest matching photos or documents, the same idea behind vector databases.
Strengths, weaknesses and when to use it
Strengths
- No training step; new data can be added instantly.
- Easy to explain with the actual examples that drove a decision.
Weaknesses
- Slow and memory-hungry at prediction time on big datasets.
- Depends heavily on a good distance measure and suffers in high dimensions.
When to use itSmall to medium datasets with a meaningful notion of similarity, or as a quick, honest baseline.