Decision Tree
Predicts by asking a series of yes/no questions about the features, like a flowchart learned automatically from data.
Also CART · classification tree · regression tree
Breiman, Friedman, Olshen and Stone (CART), 1984
It is the game of 20 questions. 'Does it have fur?' Yes. 'Does it bark?' Yes. 'It is a dog!' The tree learns which questions to ask, and in which order.
Where it fits
Not shown: 1 more that build on it. See the full list below.
+All relationships as a list
Is a kind of / part of
Siblings
Techniques that apply here
How it works
A decision tree is a flowchart. At each node it asks one question about one feature, such as “is the house larger than 120 m²?”, and sends the example left or right. At the bottom (a leaf) it outputs a prediction.
How is the tree built? Greedily, one split at a time:
- Look at every feature and every possible threshold.
- Pick the split that makes the two resulting groups as “pure” as possible (mostly one class each).
- Repeat inside each group until a stopping rule hits (maximum depth, minimum samples per leaf).
The result carves the feature space into rectangles, each with its own prediction.
Left unchecked, the tree keeps splitting until each leaf holds a single training example: perfect on training data, poor on anything new. That is overfitting, and it is why trees are usually pruned, kept shallow, or combined into ensembles such as random forests and gradient boosting.
The math
Gini impurity of a node with class proportions :
A split is scored by how much it lowers the size-weighted impurity of the children compared with the parent.
Everyday use cases
- Case 01
Loan approval flowchart
A bank's tree might ask: income above 40k? Existing debt below 30%? Each path is a rule a person can read and check.
- Case 02
Triage
Simple trees help decide which patients need urgent attention based on a few vital signs.
Strengths, weaknesses and when to use it
Strengths
- Easy to read and explain, at least when small.
- Handles numbers and categories, no feature scaling needed.
- Captures non-linear relationships and interactions.
Weaknesses
- Deep trees overfit badly.
- Unstable, a small change in data can produce a very different tree.
When to use itWhen you need rules a human can audit, or as the building block for random forests and gradient boosting.
Compare with
Check yourself
- Q01
How does a decision tree choose which question to ask first?
Show answer
B: The split that best separates the classes (largest impurity reduction)
- Q02
What happens if you let a tree grow until every leaf is pure?
Show answer
A: It usually overfits