Gated Recurrent Unit (GRU)
A simplified LSTM with two gates instead of three and no separate memory cell. Often just as accurate, with fewer parameters.
Also GRU · gated recurrent unit · update gate · reset gate
Cho et al., 2014
An LSTM is a notebook with three switches: what to erase, what to write, what to read out. A GRU makes do with two switches and is a bit quicker to use, and usually works about as well.
Where it fits
+All relationships as a list
Is a kind of / part of
Evolved from
How it works
The LSTM fixed the forgetfulness of plain RNNs with three gates and a separate memory cell. In 2014, Cho and colleagues proposed a leaner design: the gated recurrent unit.
It has just two gates and one state vector:
- The update gate decides how much of the old state to keep versus replace with new information. When it stays near zero, the state (and the gradient) passes through unchanged, which avoids vanishing gradients.
- The reset gate decides how much of the past to consider when proposing the new content.
In practice GRUs and LSTMs perform similarly; which one wins depends on the task. GRUs are a little cheaper. Both have largely been replaced by Transformers for language, and newer state space models revisit the idea of an efficient recurrent state.
Everyday use cases
- Case 01
On-device sequence models
Keyword spotting or sensor data on small devices, where fewer parameters help.
- Case 02
Early neural machine translation
The GRU was introduced in the encoder-decoder paper that helped start neural translation.
Strengths, weaknesses and when to use it
Strengths
- Fewer parameters and slightly faster than an LSTM; strong at modeling longer dependencies than plain RNNs.
Weaknesses
- Still sequential, so it trains slower than Transformers on long sequences.
When to use itSmall sequence tasks or constrained hardware where an RNN is preferred over a Transformer.
Compare with
Check yourself
- Q01
How many gates does a GRU have?
Show answer
B: 2