Hidden Markov Model
A probabilistic model of sequences in which an unseen state changes step by step and each state emits an observation you can see.
Also HMM · Viterbi algorithm · forward-backward algorithm · Baum-Welch
Baum and colleagues, 1960s; popularized for speech by Rabiner, 1989
You can't see the weather in another city, but you can see whether your friend there carries an umbrella each day. From the umbrella pattern, you can guess the most likely sequence of sunny and rainy days.
Where it fits
+All relationships as a list
How it works
Many things we observe are driven by causes we cannot see: the sounds of speech by the words being said, the umbrellas by the weather, the letters of DNA by whether they are part of a gene. A hidden Markov model captures that.
It has two parts:
- Hidden states that change over time. The next state depends only on the current one (the Markov assumption), via transition probabilities.
- Observations emitted by each state with some probability.
Three classic questions, each with an efficient algorithm:
- How likely is this observation sequence? (forward algorithm)
- What hidden-state sequence most likely produced it? (Viterbi)
- What transition and emission probabilities best explain a set of sequences? (Baum-Welch, a form of expectation-maximization, like in Gaussian mixture models)
For decades HMMs were the backbone of speech recognition. Recurrent networks and then Transformers replaced them because they can carry much richer context than a single discrete state.
Everyday use cases
- Case 01
Classic speech recognition
From the 1980s until around 2010, most speech recognizers modeled phonemes as HMM states with Gaussian mixture emissions.
- Case 02
Gene finding
Labeling stretches of DNA as gene or non-gene from the sequence of letters.
Strengths, weaknesses and when to use it
Strengths
- Principled, efficient algorithms for inference and learning; works with little data.
Weaknesses
- The Markov assumption (the next state depends only on the current one) cannot capture long-range context.
- Largely replaced by RNNs and Transformers for speech and language.
When to use itFor small sequence problems with a clear hidden-state story, or as a teaching model for sequence modeling.
Compare with
Check yourself
- Q01
What does the Viterbi algorithm find?
Show answer
A: The most likely sequence of hidden states