EngineersofAI LogoCoursesBlogResearch LabAI LettersThe LabCode BankInteractive 3DKodrEarnestJobs
Skip to main content
Interactive 3D/Entropy Explorer
Distribution
Symbols k4
Presets
Metrics
H(X) 2.0000 bits
Max H 2.0000 bits
Efficiency 100.0%
Self-info 2.000 bits
What is entropy?
H(X) measures average surprise. A certain outcome gives 0 bits. A fair coin flip gives 1 bit. Drag bars to reshape - bars colored by entropy contribution.

Entropy Explorer - Interactive Visualization

Shannon entropy H(X) = -Σ p_i log₂(p_i) measures uncertainty in bits. Maximum entropy occurs with a uniform distribution (maximum uncertainty). Zero entropy occurs when one outcome is certain. In ML, entropy appears in cross-entropy loss, decision tree splitting criteria (information gain), and feature selection. This explorer lets you sculpt a distribution and watch entropy update live.

  • Drag probability bars to change the distribution
  • Watch H(X) in bits update as you sculpt the distribution
  • See that uniform distribution maximizes entropy
  • Use presets: Uniform, Peaked (certain), Bimodal
  • Connect to cross-entropy loss, decision trees, and information gain

Part of the EngineersOfAI Interactive 3D - free interactive visualizations covering every major concept in machine learning and AI engineering. Hover any element for a plain-English explanation. No code required.