THINK++ — Engineering Visualized

LSTM & GRU — memory networks, visualized

Both are recurrent neural networks (RNNs) that process sequences one token at a time, maintaining a memory of what came before. Watch how gates control what gets remembered or forgotten.

Processing sequence

Thecatsatonthemat

Step 1/6

LSTM — Long Short-Term Memory

Forget

12%

erase old memory

Input

82%

add new info

Cell

12%

long-term memory

Output

38%

what to reveal

LSTM uses a dedicated cell state — a highway of long-term memory flowing through the sequence. Three gates act as filters:

Forget gate — "should I erase something from long-term memory?" (sigmoid → 0 means forget, 1 means keep)
Input gate — "should I write new information in?" (controls how much of the candidate enters the cell)
Output gate — "what part of memory should I reveal?" (the filtered cell becomes the hidden state h_t)

Loading Lab...

LSTM GRU

LSTM GRU

LSTM & GRU — memory networks, visualized

LSTM & GRU — memory networks, visualized