Linear Models
Models that predict from a weighted sum of the input features. Simple, fast, interpretable, and the starting point for nearly everything else.
Also generalized linear models · GLM
Each clue gets a number of points, positive or negative. Add up the points and you get your answer. Training is figuring out how many points each clue is worth.
Where it fits
Not shown: 1 more kinds and parts. See the full list below.
+All relationships as a list
Is a kind of / part of
Kinds and parts
How it works
A linear model scores an example by multiplying each feature by a weight and adding everything up. What happens next depends on the task:
- Use the score directly as a number: linear regression.
- Squash the score into a probability between 0 and 1: logistic regression.
A single artificial neuron is also a linear model followed by an activation, which is why linear models are the natural bridge to neural networks.
Everyday use cases
- Case 01
Interpretable scoring
Credit scores and medical risk scores are often linear models, so each factor's contribution can be explained.
Strengths, weaknesses and when to use it
Strengths
- Training is convex, so there are no bad local minima.
- Weights are directly interpretable.
Weaknesses
- Only straight-line (or flat-plane) relationships unless you add engineered features.
When to use itAs a first baseline, and whenever explaining the prediction matters as much as accuracy.