EngineersofAI LogoCoursesBlogResearch LabAI LettersThe LabCode BankInteractive 3DKodrEarnestJobs
Skip to main content
Interactive 3D/RNN Unrolled Through Time
HUD
t = - / 5
Gradient flows: t → 0
RNN Unrolled
Recurrent Neural Network through time steps
Sequence Length T
T = 6 steps6
Hidden Size
Input Mode
Actions
How It Works
At each time step t, the RNN computes:

h_t = tanh(Wx·x_t + Wh·h_{t-1} + b)
y_t = tanh(mean(h_t))

The hidden state h_t carries memory from previous steps. The recurrent arrow h_t → h_{t+1} is what gives the RNN its memory.
Legend
Input x_t (blue)
Hidden h_t (indigo heatmap)
Output y_t (green)

RNN Unrolled Through Time - Interactive Visualization

A recurrent neural network processes sequences by maintaining a hidden state that is updated at every time step using the same shared weight matrix. Unrolling the RNN means drawing one copy of the network per time step, making the computation graph explicit. Backpropagation through time multiplies gradients repeatedly through this unrolled graph - and when the weight matrix has eigenvalues less than 1, gradients vanish exponentially over long sequences.

  • Watch the hidden state update at each time step as new input arrives - the memory mechanism of RNNs made visual
  • See backpropagation through time: gradients flow backward through every time step simultaneously
  • Understand the vanishing gradient problem: gradient magnitude after 20 steps plotted on a log scale - it approaches zero
  • Compare gradient norms at time step 1 vs time step 50 - this is why RNNs struggle with long-range dependencies
  • Learn why shared weights across time steps make RNNs parameter-efficient but gradient flow problematic

Part of the EngineersOfAI Interactive 3D - free interactive visualizations covering every major concept in machine learning and AI engineering. Hover any element for a plain-English explanation. No code required.