EngineersofAI LogoCoursesBlogResearch LabAI LettersThe LabCode BankInteractive 3DKodrEarnestJobs
Skip to main content
Interactive 3D/Gradient Descent on a Loss Surface
Loading 3D scene…
Valleys Surface
Drag to orbit · Scroll to zoom
Learning Rate
lr = 0.05
0.010.50
Surface
Start Position
Controls
Metrics
Loss (z): -
Steps: 0
About
The ball represents your model's parameters. Gradient descent moves it in the direction that reduces loss (downhill) by learning rate × slope.
Try: Set surface to "Valleys", crank lr to 0.3, and watch it overshoot. Then lower to 0.02 and see slow but stable descent.

Gradient Descent on a Loss Surface - Interactive Visualization

Gradient descent is the engine behind every neural network. This 3D visualization shows a loss surface - a landscape where every point represents a different set of model parameters. The ball represents your model, and gradient descent rolls it downhill step by step, guided by the gradient (slope) of the loss.

  • See how learning rate controls step size - too large diverges, too small converges slowly
  • Compare Bowl (convex, single minimum), Valleys (multiple local minima), and Saddle point surfaces
  • Understand why initialization matters: different starting points reach different minima
  • See the gradient path - the trail of parameter updates the optimizer took
  • Used in every neural network training loop: SGD, Adam, AdamW all follow this principle

Part of the EngineersOfAI Interactive 3D - free interactive visualizations covering every major concept in machine learning and AI engineering. Hover any element for a plain-English explanation. No code required.