EngineersofAI LogoCoursesBlogResearch LabAI LettersThe LabCode BankInteractive 3DKodrEarnestJobs
Skip to main content
Interactive 3D/Dropout Regularization
Subnetwork #1 - p=0.30
Dropout
Randomly zero out hidden nodes each forward pass
Actions
Dropout Rate
p = 0.30Low
0 (none)0.9 (heavy)
Network Size
Layer 1 nodes6
Layer 2 nodes6
410
Active Nodes
Hidden 14/6 active
Hidden 24/6 active
Effective ensemble:
212 = 4,096 possible subnetworks
8/12 hidden nodes active now
Dropout prevents co-adaptation: nodes can't rely on specific neighbors always being there. At test time, all nodes are active but outputs scaled by (1-p).

Dropout Regularization - Interactive Visualization

Dropout randomly zeros out a fraction of neurons during each training forward pass, forcing the network to learn features that are useful even when other neurons are absent. At test time, all neurons are active but their outputs are scaled by the keep probability (inverted dropout). This is equivalent to averaging predictions across an exponential number of thinned subnetworks.

  • See the random binary mask applied to each layer - different neurons dropped every forward pass
  • Understand inverted dropout: weights are scaled by 1/keep_prob during training so no adjustment is needed at test time
  • Watch how dropout prevents co-adaptation: neurons cannot rely on specific other neurons always being present
  • Compare training curves with and without dropout - the training loss is higher but the validation gap closes significantly
  • Learn why dropout works as implicit ensemble averaging across 2^N subnetworks of the full network

Part of the EngineersOfAI Interactive 3D - free interactive visualizations covering every major concept in machine learning and AI engineering. Hover any element for a plain-English explanation. No code required.