Skip to content
ML Atlas

TD-Gammon

A neural network that taught itself backgammon by playing against itself with temporal-difference learning and reached a level close to the best human players.

Also Tesauro · backgammon

Fig. / How it works
TD-GammonTD-GammonTD-GammonTD-GammonMOVESWHO WONIT LEARNS FROM GAMES AGAINST ITSELF
Role · kind
Real systemsFamous system
Level
Beginner

Gerald Tesauro, IBM Research, 1992

A computer program that started knowing nothing about backgammon strategy, played itself over a million times, and by learning from wins and losses became nearly as good as world-class players.

Where it fits

Reinforcement LearningReinforcement LearningMultilayer PerceptronMultilayer PerceptronQ-LearningQ-LearningSelf-PlaySelf-PlayTD-Gammon
+All relationships as a list

Is a kind of / part of

See it on the full map

How it works

In the early 1990s, Gerald Tesauro at IBM trained a neural network to play backgammon using only games against itself. TD-Gammon became one of the first big successes of reinforcement learning.

The setup was simple:

  • A small neural network looked at the board and estimated the chance of winning.
  • It chose moves by picking the position it rated best.
  • After each move, it nudged its earlier estimate toward its new one, and at the end toward the actual result. This is temporal-difference (TD) learning, the same idea behind Q-learning.

Starting from no strategic knowledge, it reached strong intermediate play. With some hand-designed input features and more self-play, version 2.1 trained on 1.5 million games and played just below the level of the world’s best humans. Experts even adopted some of its unusual opening moves.

Its contrast with Deep Blue is instructive: Deep Blue searched with hand-tuned rules, TD-Gammon learned its judgment. That learned-intuition idea came back at scale in AlphaGo and AlphaZero.

Everyday use cases

  • Case 01

    How you've probably met it

    As the classic example in reinforcement learning textbooks, and indirectly through the backgammon opening moves it popularized.

Strengths, weaknesses and when to use it

Strengths

  • Showed that a neural network could learn expert-level play from self-play alone.

Weaknesses

  • Backgammon's dice make exploration easy; the same simple recipe failed in many other games until much later.

When to use itAs the historical ancestor of AlphaGo-style self-play learning.

Compare with

Go deeper

Sources

  1. [1]Tesauro, Temporal Difference Learning and TD-Gammon, Communications of the ACM 38(3) (1995)
  2. [2]Sutton and Barto, Reinforcement Learning: An Introduction (2nd ed.), section 16.1

Facts checked as of 2026-10-12