Speech Recognition
Turning spoken audio into written text. Modern systems read a spectrogram and write out the words with a neural network.
Also ASR · automatic speech recognition · speech-to-text · transcription
A very fast typist who listens to you talk and writes down every word, even with background noise and accents.
Where it fits
+All relationships as a list
Is a kind of / part of
Kinds and parts
Siblings
Uses / built on
Used by
How it works
Speech recognition has to bridge a big gap: from a fast-changing pressure wave to a short string of words. Today that bridge is usually one neural network:
- Convert the audio to a spectrogram.
- An encoder (often a Transformer) reads the spectrogram and builds a representation of the sounds.
- A decoder writes out text tokens one at a time, like a language model that is listening.
Older systems were pipelines of separate parts (sound units, pronunciation dictionaries, word statistics), each tuned by hand. End-to-end training on huge amounts of transcribed audio replaced most of that.
Because the decoder is a language model, it uses context: “I scream” and “ice cream” sound almost the same, and the surrounding words decide. The flip side is that it can produce fluent text that was not actually said, especially when the audio is silent or musical, the speech version of hallucination.
Everyday use cases
- Case 01
Dictation and voice typing
Speak a message instead of typing it.
- Case 02
Subtitles and meeting notes
Automatic captions for videos and transcripts of calls.
- Case 03
Voice assistants
The first step of every spoken command is recognizing the words.
Famous examples
- 2022Whisper
OpenAI's open-source speech recognition model, trained on 680,000 hours of audio from the web, that transcribes and translates speech in many languages.
- 2011Voice Assistants
Spoken assistants such as Siri, Alexa and Google Assistant that chain wake-word detection, speech recognition, language understanding and text-to-speech.
Strengths, weaknesses and when to use it
Strengths
- Hands-free, accessible input.
- Modern models handle many languages and accents.
Weaknesses
- Errors on rare names, jargon and heavy noise.
- Can occasionally invent words, especially on silence or music.
When to use itWhenever speech needs to become searchable, editable text.
Compare with
Check yourself
- Q01
What does a modern speech recognizer usually take as input?
Show answer
B: A log-Mel spectrogram of the audio