How is a GRU different from an LSTM?

Question

AcadiFi · Accepted Answer

Gated Recurrent Unit (GRU), proposed by Cho et al. (2014), simplifies LSTM by merging the forget and input gates into a single update gate and eliminating the separate cell state.

GRU equations:
- **Update gate** z_t = sigma(W_z [h_{t-1}, x_t] + b_z): balances old state vs new candidate.
- **Reset gate** r_t = sigma(W_r [h_{t-1}, x_t] + b_r): how much past to forget when computing candidate.
- Candidate: h_tilde_t = tanh(W [r_t * h_{t-1}, x_t] + b)
- New state: h_t = (1 - z_t) * h_{t-1} + z_t * h_tilde_t

Two gates instead of three; no separate cell state. Roughly 25% fewer parameters than LSTM.

When to use which:
- **GRU**: smaller datasets, shorter sequences, when compute is constrained. Tends to converge faster.
- **LSTM**: very long sequences, when you need the extra expressiveness of separate cell state and gating. Often more robust on complex time-series.

Empirical evidence: Chung et al. (2014) showed GRU and LSTM perform comparably on most tasks; the choice is often a hyperparameter tuned by cross-validation. For finance applications like intraday price prediction with 390-minute windows, GRU's faster training is a practical advantage.

Implementation note: in PyTorch, `nn.GRU` and `nn.LSTM` are drop-in replacements - just swap the class. Both support bidirectional and stacked configurations.

Key CFA takeaway: GRUs and LSTMs are both gated RNN variants that address vanishing gradients. GRU is simpler, LSTM is more expressive. Either is preferred over vanilla RNN for finance sequences.

How is a GRU different from an LSTM?

Master Level II with our CFA Course

Related Questions

Practice Questions