Skip to content

AlexNet

The deep convolutional network that won the 2012 ImageNet challenge by a huge margin and kicked off the deep learning boom.

Fig. / How it works
IMAGEFEATURE MAPA 3×3 FILTER SLIDES OVER THE IMAGE
Role · kind
Real systemsFamous system
Level
Intermediate

Krizhevsky, Sutskever and Hinton, University of Toronto, 2012

In 2012 there was a big contest to name the objects in a million photos. A team showed up with a neural network trained on gaming graphics cards and beat everyone else by a mile. After that, everyone switched to neural networks.

Where it fits

+All relationships as a list

Is a kind of / part of

Trained with

Evolved into

See it on the full map

How it works

Before 2012, the best image recognition systems used features designed by hand by vision experts. AlexNet learned its features directly from pixels, with a convolutional neural network trained on 1.2 million labeled ImageNet photos.

Several ingredients came together:

  • GPUs: training ran on two consumer graphics cards, making a network this size practical.
  • ReLU activations: much faster training than sigmoid or tanh.
  • Dropout and data augmentation (crops, flips) to fight overfitting.

The result: a top-5 error of 15.3% in the ILSVRC-2012 competition, against 26.2% for the next best entry. The gap was so large that the computer vision community moved to deep learning almost overnight, and the rest of AI followed.

Everyday use cases

  • Case 01

    How you've probably met it

    Indirectly, everywhere. The photo search, face grouping and visual search on your phone descend from the shift AlexNet started.

Strengths, weaknesses and when to use it

Strengths

  • Proved that deep networks plus GPUs plus big data beat hand-engineered vision features.

Weaknesses

  • Long since surpassed in accuracy and efficiency by later architectures such as ResNet and Vision Transformers.

When to use itAs a historical reference point; not as a model to deploy today.

Go deeper