EngineersofAI LogoCoursesBlogResearch LabAI LettersThe LabCode BankInteractive 3DKodrEarnestJobs
Skip to main content
Interactive 3D/Batch Normalization Effect
Press "Apply BN" →
Batch Normalization
Shifts activations to zero-mean, unit variance
Actions
Batch Size
N = 64
4128
Raw Activations
Mean3.0
-5+5
Std Dev4.0
0.110
BN Parameters
γ (scale)1.00
0.13
β (shift)0.00
-3+3
Stats
Pre-BN
μ = 2.744
σ = 3.925
Post-BN
μ = 0.000
σ = 1.000
Learned
γ = 1.00
β = 0.00
BN normalizes each mini-batch to zero mean, unit variance during training. γ and β let the network undo normalization if needed - they are learned via backprop.

Batch Normalization Effect - Interactive Visualization

Batch normalization normalizes the inputs to each layer so that they have zero mean and unit variance within each mini-batch. This reduces internal covariate shift - the phenomenon where the distribution of each layer's inputs keeps changing during training, forcing later layers to constantly re-adapt. Learnable scale (gamma) and shift (beta) parameters let the network recover any normalization it needs to undo.

  • See the distribution of activations before and after batch norm - the shift from arbitrary distributions to centered, unit-variance outputs
  • Understand internal covariate shift: without normalization, later layers receive a moving target as earlier layers update
  • Watch how batch norm acts as a regularizer: the noise from batch statistics slightly perturbs gradients, reducing overfitting
  • Compare Batch Norm vs Layer Norm vs Group Norm - when each is appropriate (CNNs vs transformers vs small batches)
  • Understand the gamma and beta parameters: they allow the network to scale and shift post-normalization to any distribution it needs

Part of the EngineersOfAI Interactive 3D - free interactive visualizations covering every major concept in machine learning and AI engineering. Hover any element for a plain-English explanation. No code required.