AlphaGo
DeepMind's Go program that combined deep neural networks with tree search and, in March 2016, beat top professional Lee Sedol 4 games to 1.
Also Lee Sedol
Silver et al., DeepMind, 2016
Go was thought too intuitive for computers. AlphaGo first studied human games, then practiced against itself, and used that intuition to guide a careful look-ahead. It beat one of the best players in the world.
Where it fits
+All relationships as a list
Is a kind of / part of
Siblings
Uses / built on
Evolved into
How it works
Go has more possible board positions than there are atoms in the observable universe, and good positions are hard to judge, so the brute-force search that worked for chess was not enough.
AlphaGo combined three ideas:
- A policy network (a CNN) that suggests promising moves. It was first trained with supervised learning to predict human expert moves, then improved by playing against itself using policy gradients.
- A value network that estimates who is winning from a position.
- Monte Carlo tree search guided by both networks: the policy network narrows which moves to explore, the value network judges where the search ends.
The 2016 Nature paper reported a 5-0 win over European champion Fan Hui. In March 2016, AlphaGo beat Lee Sedol, one of the strongest players of all time, 4-1. Later versions, AlphaGo Zero and AlphaZero, dropped the human games entirely and learned from self-play alone.
Everyday use cases
- Case 01
How you've probably met it
Through the news and the documentary about the 2016 match; its successors' ideas now influence Go training and research far beyond games.
Strengths, weaknesses and when to use it
Strengths
- Showed learning plus search could master a domain long considered out of reach.
Weaknesses
- Required huge compute and was specific to Go; later versions (AlphaGo Zero, AlphaZero) removed the need for human games.
When to use itAs a landmark example of combining deep learning, reinforcement learning and search.