Skip to content
ML Atlas

Audio Classification

Labeling sounds: is this a dog bark, glass breaking, a siren, a song, or the wake word 'Hey Siri'? Usually done by treating a spectrogram like an image.

Alsosound classification · sound event detection · keyword spotting · wake word detection · AudioSet

Fig. / How it works
SoundSoundSpectrogramSpectrogramCNNCNNLabel: "dog bark"Label: "dog bark"
Role · kind
Using and applyingConcept
Level
Beginner

Your phone listens to a short clip and says 'that's a baby crying' or 'that's a doorbell', like a friend who is very good at naming sounds.

Where it fits

Speech and AudioSpeech and AudioConvolutional Neural NetworkConvolutional Neural…SpectrogramSpectrogramVoice AssistantsVoice AssistantsAudio Classification
+All relationships as a list

Is a kind of / part of

See it on the full map

Try it

See what sound looks like
Loading interactive widget…
TakeawayA spectrogram turns sound into a picture of pitch over time, which is what speech and audio models actually read. Window size trades sharp timing against sharp pitch.

How it works

Turn a sound into a spectrogram and it becomes a picture: time across, frequency up, loudness as brightness. A dog bark, a siren and a guitar chord each leave a recognizable pattern. That means the whole toolkit of image classification applies.

Typical uses:

  • Keyword spotting: smart speakers and phones run a tiny, always-on model that only listens for the wake word, on a low-power chip (edge AI).
  • Sound event detection: smoke alarms, breaking glass, coughing; with timestamps for when they happen.
  • Music tagging: genre, mood, instruments.
  • Nature monitoring: identifying bird and whale calls from field recordings.

Models are usually CNNs or audio Transformers, often pretrained on Google’s AudioSet (millions of labeled YouTube clips) and then fine-tuned with transfer learning.

Try the spectrogram widget to see how different sounds look.

Everyday use cases

  • Case 01

    Wake words

    A tiny model on your smart speaker listens only for its wake word before anything is sent anywhere.

  • Case 02

    Sound recognition on phones

    Accessibility features alert deaf or hard-of-hearing users to doorbells, alarms or a crying baby.

Strengths, weaknesses and when to use it

Strengths

  • Reuses mature image-classification techniques; small models can run on very low power.

Weaknesses

  • Background noise and overlapping sounds are hard; labeled audio data is scarcer than images.

When to use itAny task that needs to recognize what kind of sound is present.

Compare with

Check yourself

  1. Q01

    Why can image models like CNNs classify audio?

    Show answer

    A: Audio can be turned into a spectrogram, which is an image of frequencies over time

Go deeper

Sources

  1. [1]Gemmeke et al., Audio Set: An ontology and human-labeled dataset for audio events (ICASSP 2017)
  2. [2]Gong, Chung and Glass, AST: Audio Spectrogram Transformer (2021)