Tree-Based Models
Models built from decision trees, a series of yes/no questions about the features, from a single readable tree to forests and boosted ensembles of hundreds.
Also decision tree methods · tree ensembles · tree models
A game of 20 questions: is it bigger than a breadbox? Does it live in water? Each answer narrows things down until you can name it. Tree models learn which questions to ask.
Where it fits
+All relationships as a list
Is a kind of / part of
Kinds and parts
How it works
Everything in this family starts with a decision tree: a flowchart of yes/no questions about the features, learned from data.
One tree is easy to read but unstable: change a few rows and it can grow very differently. Combining many trees fixes that, which is where ensemble methods come in:
- Random forests and bagging average many trees trained on random resamples.
- Gradient boosting and AdaBoost add small trees one by one, each fixing the previous ones’ mistakes.
- Isolation forests use random trees for a different job: spotting anomalies.
On tabular data, tree ensembles still often beat neural networks, and they are usually the first thing practitioners try.
Everyday use cases
- Case 01
Credit decisions
A bank's loan rules as a readable flowchart, or a boosted ensemble for accuracy.
- Case 02
Fraud detection
Gradient-boosted trees score card transactions in milliseconds.
Strengths, weaknesses and when to use it
Strengths
- Handle mixed numeric and categorical features with little preprocessing.
- Ensembles are often the most accurate models for tabular data.
Weaknesses
- A single tree overfits easily; ensembles lose the single tree's readability.
- Poor at extrapolating beyond the range seen in training.
When to use itAlmost any tabular (spreadsheet-like) prediction problem. Start with a gradient-boosted or random forest model.