Q-Learning in GridWorld - Interactive Visualization

Q-learning is a model-free reinforcement learning algorithm that learns the value Q(s,a) of taking action a in state s. The Q-value is updated using the Bellman equation after every step: Q(s,a) += alpha * (r + gamma * max Q(s',a') - Q(s,a)). No knowledge of the environment's transition model is needed - the agent discovers optimal behavior purely through interaction and reward signals.

• Watch the agent explore the GridWorld and see Q-values update after every step - initially random, gradually converging
• Understand epsilon-greedy: with probability epsilon, take a random action; otherwise take the greedy best action
• See why pure exploitation (epsilon=0) gets stuck in local optima: the agent never discovers better paths it has not visited
• Watch the Q-table heatmap: cells brighten as Q-values increase for actions that lead toward reward
• Understand temporal difference error (TD error): the discrepancy between current estimate and bootstrapped target - this is what drives learning

Part of the EngineersOfAI Interactive 3D - free interactive visualizations covering every major concept in machine learning and AI engineering. Hover any element for a plain-English explanation. No code required.