EngineersofAI LogoCoursesBlogResearch LabAI LettersThe LabCode BankInteractive 3DKodrEarnestJobs
Skip to main content
Interactive 3D/Cross-Entropy Loss
Prediction
q̂ (predicted)0.500
Losses at q̂ = 0.500
CE loss 0.6931
MSE loss 0.2500
Gradients at q̂
∂CE/∂q̂ -2.000
∂MSE/∂q̂ -1.000
CE has larger gradient - trains faster here
Why use CE over MSE?
When q̂ is near 0 and label is 1, CE gradient −1/q̂ is enormous - pushing the model hard. MSE gradient is small. CE converges faster for classification tasks.

Cross-Entropy Loss - Interactive Visualization

Cross-entropy loss H(p, q̂) = -log(q̂) for a correct class (p=1) diverges to infinity as q̂ → 0, creating large gradients for wrong predictions. MSE loss = (1-q̂)² creates much smaller gradients near 0. This is why neural networks use cross-entropy for classification: it provides stronger gradient signal when predictions are far from correct.

  • Drag predicted probability and watch CE and MSE losses update
  • See gradient of CE vs MSE - CE is much steeper near 0
  • Understand why log(q̂) creates large gradients for confident wrong predictions
  • Compare CE surface vs MSE surface visually
  • Foundation for training classifiers with softmax output layers

Part of the EngineersOfAI Interactive 3D - free interactive visualizations covering every major concept in machine learning and AI engineering. Hover any element for a plain-English explanation. No code required.