LSTM Gates Visualization - Interactive Visualization

The LSTM (Long Short-Term Memory) cell was designed specifically to fix the vanishing gradient problem in RNNs. It maintains a separate cell state - a memory highway - that runs through time with only element-wise operations (no matrix multiplications). Three gates control what information enters, exits, and is erased from this cell state, enabling gradients to flow through hundreds of time steps without vanishing.

• See the forget gate: a sigmoid that outputs 0 to 1 for each cell state element, deciding what fraction of old memory to keep
• Watch the input gate and cell gate jointly determine what new information gets written into cell state
• Understand the output gate: it reads from cell state and controls what the hidden state exposes to the next layer
• See the gradient highway: because cell state updates are additive (not multiplicative), gradients flow back with near-constant magnitude
• Compare LSTM vs GRU - GRU has only 2 gates and merges cell and hidden state, reducing parameters by ~25%

Part of the EngineersOfAI Interactive 3D - free interactive visualizations covering every major concept in machine learning and AI engineering. Hover any element for a plain-English explanation. No code required.