Reinforcement Learning
Learning by trial and error: an agent acts in an environment, receives rewards, and gradually learns a strategy that collects as much reward as possible.
AlsoRL · agent · environment · reward
Like training a dog with treats. Nobody shows it the right answer; it tries things, and the treats tell it what worked.
Where it fits
Not shown: 8 more kinds and parts. See the full list below.
+All relationships as a list
Overview
Reinforcement learning is the third big paradigm next to supervised and unsupervised learning. There are no right answers to copy, only rewards that arrive, sometimes much later, after a sequence of actions.
The setting is formalized as a Markov decision process: an agent sees a state, picks an action, and the environment returns a reward and a new state. The agent must balance exploration and exploitation: try new things or stick with what works.
The main approaches:
- Value-based: learn how good each action is in each state, as in Q-learning and DQN.
- Policy-based: directly adjust the strategy toward actions that led to reward, as in policy gradients and PPO.
- Planning with search: look ahead with a model of the environment, as in Monte Carlo tree search. AlphaGo combined all of this.
A recurring danger is reward hacking: the agent maximizes the reward you wrote down, not the goal you meant.
Everyday use cases
- Case 01
Game-playing AI
From Atari games to Go, RL agents have reached and exceeded top human level.
- Case 02
Robotics and control
Learning to walk, grasp or balance through practice, often in simulation first.
- Case 03
Tuning chat assistants
RLHF uses reinforcement learning to make language models more helpful.