Monte Carlo Tree Search for LLM Reasoning - Interactive Visualization

OpenAI o1 and DeepSeek-R1 use search over reasoning steps at inference time, allocating more compute to harder problems. MCTS explores a tree of reasoning steps by selecting promising nodes (UCB1), expanding with new steps, simulating to completion, and backpropagating scores from a process reward model. This demo shows the four MCTS phases applied to a math reasoning problem.

• Select phase - watch UCB1 balance exploration vs exploitation to choose which reasoning branch to extend
• Expand phase - see how new candidate reasoning steps are sampled from the LLM and added to the tree
• Simulate phase - trace how a rollout from a new node completes the reasoning chain to reach a final answer
• Backpropagate phase - see how the process reward model score propagates up the tree to update node values
• Compare single-path chain-of-thought vs MCTS search on the same problem to see where search wins

Part of the EngineersOfAI Interactive 3D - free interactive visualizations covering every major concept in machine learning and AI engineering. Hover any element for a plain-English explanation. No code required.