Classical Machine Learning
The pre-deep-learning toolkit: regression, trees, ensembles, nearest neighbors and clustering. Still the best choice for most tabular data.
Alsotraditional ML · shallow learning
These are the trusty hand tools of machine learning. Not as flashy as power tools, but quick, cheap and often exactly right for the job.
Where it fits
Not shown: 11 more kinds and parts. See the full list below.
+All relationships as a list
Inside this domain
- Anomaly Detection
- Card Fraud Detection
- Clustering
- Credit Scoring
- Dimensionality Reduction
- Email Spam Filters
- Ensemble Methods
- Home Price Estimators
- Instance-Based Learning
- Kernel Methods
- Linear Models
- Probabilistic Models
- Recommender Systems
- Streaming and Video Recommendations
- Time Series Forecasting
- Tree-Based Models
Techniques that apply here
Overview
“Classical” does not mean outdated. For tabular data (rows and columns, like a spreadsheet), tree ensembles such as random forests and gradient boosting still regularly beat deep learning, train in seconds and are easier to explain.
The main groups in this domain:
- Linear models: linear regression for numbers, logistic regression for yes/no.
- Instance-based: k-nearest neighbors predicts from the most similar past examples.
- Trees and ensembles: a decision tree asks a sequence of questions; ensembles combine many trees.
- Probabilistic and kernel methods: naive Bayes and support vector machines.
- Clustering: k-means finds groups without labels.
- Dimensionality reduction: PCA compresses many columns into a few.
What they share: the features are usually designed by people, and the models are small enough to train on a laptop.
Everyday use cases
- Case 01
Credit scoring and fraud detection
Banks rely heavily on gradient boosting and logistic regression for spreadsheet-like data.
- Case 02
Customer segmentation
Clustering groups shoppers into types for marketing.
- Case 03
Quick baselines
Before training a deep network, a random forest tells you how hard the problem really is.