EngineersofAI LogoCoursesBlogResearch LabAI LettersThe LabCode BankInteractive 3DKodrEarnestJobs
Skip to main content
Interactive 3D/Process Reward Models (PRM)
Controls
Score With
Solution Type
Steps4
34
PRM Analysis
Step 10.97
Step 20.12
Step 30.08
Step 40.05
Error at step 4 - ORM misses this!
ORM rewards only correct final answers - cannot identify where reasoning went wrong. PRM scores each reasoning step, providing a dense training signal.

Process Reward Models (PRM) - Interactive Visualization

Outcome reward models (ORM) only score final answers - they cannot tell you which reasoning step was wrong. Process reward models (PRM) score every intermediate step, providing much richer training signal for reasoning tasks. PRMs guide MCTS tree search by pruning branches with low step scores, enabling models like o1 and DeepSeek-R1 to find correct reasoning paths.

  • ORM limitation: final answer scored 0 or 1, no signal on which step failed
  • PRM step scoring: each reasoning step assigned a correctness probability 0.0-1.0
  • MCTS pruning: branches with cumulative step score below threshold eliminated early
  • Search tree visualization: correct path found faster with PRM guidance vs random rollout

Part of the EngineersOfAI Interactive 3D - free interactive visualizations covering every major concept in machine learning and AI engineering. Hover any element for a plain-English explanation. No code required.