Generative vs Discriminative Models
Discriminative models learn to tell classes apart; generative models learn what the data itself looks like, so they can also create new examples.
Also generative model · discriminative model · joint distribution · conditional distribution
A discriminative student learns 'cats have pointy ears, dogs don't' just to tell them apart. A generative student learns what cats and dogs look like so well they could draw a new one.
Where it fits
+All relationships as a list
How it works
Two ways to approach the same data:
- A discriminative model learns the boundary between classes: given an input, which label? It models . Logistic regression, SVMs and most image classifiers are discriminative.
- A generative model learns how the data itself is distributed, or . Because it knows what realistic data looks like, it can sample new examples.
Classic generative models include naive Bayes, Gaussian mixtures and hidden Markov models. Modern deep generative models are what people mean by generative AI:
- Autoregressive models such as GPT generate one token at a time.
- VAEs and GANs map a random latent vector to an image.
- Diffusion models turn noise into an image step by step.
- Normalizing flows learn an invertible transformation of simple noise.
For a pure classification task with plenty of labels, discriminative models are usually more accurate. But today’s biggest generative models are so capable that they are often used as classifiers too, just by asking.
Everyday use cases
- Case 01
Spam filtering both ways
Naive Bayes models how spam and normal emails are written (generative); logistic regression learns the boundary directly (discriminative).
- Case 02
Creating content
Image and text generators are generative models of pixels and tokens.
Strengths, weaknesses and when to use it
Strengths
- Clarifies why some models can create and others can only judge.
Weaknesses
- The line blurs in practice; LLMs are generative models routinely used as classifiers.
When to use itWhen deciding whether you need to classify, to generate, or both.
Compare with
Check yourself
- Q01
Which is a discriminative model?
Show answer
A: Logistic regression
- Q02
What can a generative model do that a purely discriminative one cannot?
Show answer
A: Produce new examples that look like the data