Gaussian Mixture Model
Models data as a blend of several bell-shaped (Gaussian) clusters and assigns each point a probability of belonging to each one.
Also GMM · mixture model · expectation-maximization · EM algorithm · soft clustering
EM algorithm: Dempster, Laird and Rubin, 1977
Heights in a school mix kids and teachers. Each group alone forms a bell curve. A GMM figures out both bell curves and, for each person, how likely they are a kid or a teacher.
Where it fits
+All relationships as a list
Is a kind of / part of
Uses / built on
Evolved from
How it works
k-means puts every point in exactly one round cluster. Real groups often overlap and come in different shapes. A Gaussian mixture model handles both.
It assumes the data was generated by a few Gaussian “blobs”, each with its own center, spread and orientation, and each responsible for some share of the points. The fitting algorithm, expectation-maximization (EM), alternates two steps, much like k-means:
- Expectation: for every point, compute how likely each blob is to have produced it. These are soft assignments, like 0.7 / 0.3.
- Maximization: move each blob’s center and reshape its spread to fit the points it is responsible for, weighted by those probabilities.
Repeat until nothing changes much.
Because a GMM is a full probability model, it also tells you how unusual a point is. Points with very low likelihood under every component are candidates for anomaly detection. Pick the number of components with a criterion like BIC or with validation.
Everyday use cases
- Case 01
Customer segments with overlap
A shopper can be 70% "bargain hunter" and 30% "premium buyer" instead of being forced into one group.
- Case 02
Speaker modeling
For decades, speaker recognition systems modeled each voice's sound features as a GMM.
Strengths, weaknesses and when to use it
Strengths
- Soft, probabilistic assignments and elliptical clusters of different sizes.
- Gives a full density, useful to flag unlikely points.
Weaknesses
- Must choose the number of components; EM can get stuck in poor local optima.
- Struggles with non-elliptical shapes and very high dimensions.
When to use itWhen clusters overlap or have different shapes and sizes, and you want probabilities rather than hard labels.
Compare with
Check yourself
- Q01
How does a GMM assignment differ from k-means?
Show answer
A: It gives each point a probability for each cluster
- Q02
What algorithm usually fits a GMM?
Show answer
B: Expectation-maximization