Random Forest
Many decision trees, each trained on a random sample of the data and features, vote together. Robust, accurate and hard to break.
Also RF · random forests · random decision forest
Breiman, 2001
Instead of trusting one doctor, you ask a hundred doctors who each studied slightly different patients, and go with the majority opinion.
Where it fits
+All relationships as a list
Is a kind of / part of
Siblings
Uses / built on
How it works
A single decision tree is accurate on its training data but jittery: change a few rows and it changes shape. A random forest turns that weakness into a strength.
- Bootstrap: for each tree, draw a random sample of the training rows with replacement. Each tree sees a slightly different dataset.
- Random features: at every split, a tree may only consider a random subset of the features. This stops every tree from splitting on the same dominant feature first, so the trees really differ.
- Vote: to predict, run the example through every tree and take the majority vote (or the average for numbers).
Each tree overfits in its own way, and the errors largely cancel out. Adding trees never makes the forest overfit more; it just stops helping after a few hundred.
A free bonus: each tree skipped roughly a third of the rows during training. Testing each tree on its skipped rows gives an out-of-bag error estimate without a separate validation set.
Everyday use cases
- Case 01
Medical diagnosis support
Many "doctor" trees vote on whether a patient's measurements suggest a condition.
- Case 02
Land-cover mapping
Classifying satellite image pixels as forest, water, city or farmland.
- Case 03
Feature importance
Forests report which inputs mattered most, a quick way to explore a new dataset.
Strengths, weaknesses and when to use it
Strengths
- Strong accuracy with almost no tuning.
- Resistant to overfitting as you add trees.
- Gives feature importance scores and an out-of-bag error estimate for free.
Weaknesses
- Large and slower to predict than a single tree.
- Loses the readability of a single tree.
When to use itAs your go-to first serious model for tabular data.
Compare with
Check yourself
- Q01
What are the two sources of randomness in a random forest?
Show answer
B: Bootstrap samples of rows and random subsets of features at each split
- Q02
What usually happens as you add more trees?
Show answer
A: Accuracy improves then levels off; it does not overfit from more trees