MDP & Value Iteration GridWorld - Interactive Visualization

A Markov Decision Process is the mathematical framework underlying all reinforcement learning. It defines states, actions, transition probabilities, and rewards. Value iteration applies the Bellman optimality equation repeatedly: the value of a state equals the reward plus the discounted value of the best next state. After enough iterations, values converge and the optimal policy can be read off directly from the value function.

• Place walls, positive rewards (gold), and negative rewards (traps) to design your own GridWorld environment
• Watch value iteration propagate values backward from the reward: initially only the goal cell has nonzero value
• Adjust the discount factor gamma and see how shorter-sighted agents (low gamma) behave differently from far-sighted ones
• See the optimal policy arrows emerge as values converge - each state points toward its best next move
• Understand the Bellman equation: V*(s) = max_a [R(s,a) + gamma * sum_s P(s'|s,a) * V*(s')] made visual

Part of the EngineersOfAI Interactive 3D - free interactive visualizations covering every major concept in machine learning and AI engineering. Hover any element for a plain-English explanation. No code required.