KL Divergence - Interactive Visualization

KL divergence D_KL(P‖Q) = Σ p_i log(p_i/q_i) measures how much information is lost when using Q to approximate P. It is not symmetric: D_KL(P‖Q) ≠ D_KL(Q‖P). Forward KL penalizes placing zero mass where P has mass (zero-avoiding). Reverse KL causes mode-seeking. These differences matter for VAEs, RL, and distribution matching.

• See KL(P‖Q) and KL(Q‖P) computed simultaneously to show asymmetry
• Adjust μ and σ for both distributions independently
• See what happens when Q=0 but P>0 (KL → ∞)
• Understand forward vs reverse KL and their different behaviors
• Foundation for VAE ELBO, RL policy optimization, and model distillation

Part of the EngineersOfAI Interactive 3D - free interactive visualizations covering every major concept in machine learning and AI engineering. Hover any element for a plain-English explanation. No code required.