EngineersofAI LogoCoursesBlogResearch LabAI LettersThe LabCode BankInteractive 3DKodrEarnestJobs
Skip to main content
Interactive 3D/Mutual Information
Correlation
ρ (correlation)0.60
n points200
Entropy Decomposition
H(X) 0.000 bits
H(Y) 0.000 bits
H(X,Y) 0.000 bits
I(X;Y) 0.000 bits
H(X|Y) 0.000 bits
H(Y|X) 0.000 bits
Mutual information
I(X;Y) measures how much knowing X reduces uncertainty about Y. At ρ=0 it's ~0. At ρ=±1 it peaks. The Venn diagram shows the overlap.

Mutual Information - Interactive Visualization

Mutual information I(X;Y) = H(X) + H(Y) - H(X,Y) measures how much knowing X reduces uncertainty about Y. Unlike correlation, it captures nonlinear dependence. I(X;Y) = 0 iff X and Y are independent. This visualization shows correlated scatter data and the entropy Venn diagram, where the overlap is I(X;Y).

  • Adjust correlation ρ from -1 to 1 and watch I(X;Y) change
  • See entropy Venn diagram: H(X), H(Y), I(X;Y) overlap
  • Understand I(X;Y) = 0 means statistical independence
  • Compare to correlation: MI captures nonlinear dependence too
  • Foundation for feature selection, InfoGAN, and CPC representation learning

Part of the EngineersOfAI Interactive 3D - free interactive visualizations covering every major concept in machine learning and AI engineering. Hover any element for a plain-English explanation. No code required.