The Evolution of AI Agents
From "think step by step" to autonomous systems that write production code.
AI agents went from a prompting trick (Chain-of-Thought) to production systems that autonomously browse the web, write code, and coordinate with other agents — in roughly two years. Each paper below is a step in that progression, and each one fixes a specific limitation of the previous approach.
Read in order. Each paper fixes a failure mode exposed by the previous one. Click any card to collapse it.
Chain-of-Thought — The Foundation
Wei, Wang, Schuurmans et al. (Google Research)
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Shows that prompting LLMs to produce intermediate reasoning steps ('think step by step') dramatically improves performance on arithmetic, commonsense, and symbolic reasoning tasks. The reasoning chain is generated as plain text before the final answer.
Key insight: The model already knows how to reason — it just needs to be shown that it should reason out loud. This is the prerequisite for every agent paper that follows: if the model can't reason, it can't plan, and if it can't plan, it can't act.
ReAct — Reasoning + Tool Use
Yao, Zhao, Yu et al. (Princeton / Google)
ReAct: Synergizing Reasoning and Acting in Language Models
Interleaves chain-of-thought reasoning with tool calls in a Thought → Action → Observation loop. The model generates thoughts, calls tools (search, calculator, code), observes results, and thinks again — iteratively — until it reaches an answer.
Key insight: ReAct is not a framework. It's a prompting pattern. Everything called an 'agent framework' today is essentially ReAct plus tooling infrastructure. Understanding this paper means understanding 80% of what agent frameworks do.
Engineering note: Always set max_steps. The model can get stuck in a reasoning loop. Also: context overflow is the primary production failure mode for long agent runs — summarise history periodically.
Toolformer — Self-Supervised Tool Use
Schick, Dwivedi-Sahu, Dessì et al. (Meta AI)
Toolformer: Language Models Can Teach Themselves to Use Tools
Fine-tunes a model to autonomously decide when and how to call tools, using a self-supervised approach: the model generates candidate tool calls, executes them, and keeps only the calls that reduce perplexity on future tokens. No human annotation required.
Key insight: ReAct tells the model to use tools. Toolformer teaches the model to decide when. The distinction matters in production: a model that over-calls tools is slow and expensive; a model that knows when tools add value is efficient.
Gorilla — API-Aware LLMs
Patil, Zhang, Wang, Gonzalez (UC Berkeley)
Gorilla: Large Language Model Connected with Massive APIs
Fine-tunes LLMs on a large dataset of API documentation and usage examples. The model learns to generate correct, executable API calls — including correct parameter names, types, and values — rather than hallucinating plausible-looking but wrong calls.
Key insight: API hallucination is the primary failure mode when LLMs call real tools. Gorilla shows that fine-tuning on API docs is more reliable than prompt engineering. For production agents with many tools, this approach significantly reduces tool call errors.
MemGPT — Hierarchical Memory Management
Packer, Fang, Patil et al. (UC Berkeley)
MemGPT: Towards LLMs as Operating Systems
Treats the LLM's context window like CPU registers and implements a virtual memory hierarchy: in-context storage (active), external storage (files/DBs), and archival storage (indexed). The model manages its own memory — explicitly moving information in and out of context.
Key insight: An agent that forgets what it did three hours ago cannot complete long tasks. MemGPT is the first serious treatment of agent memory as a systems problem, not a prompting problem. The OS analogy is deliberate and useful.
Engineering note: Context management adds overhead and complexity. Start with simple summarisation before implementing full hierarchical memory. MemGPT is most valuable for agents with sessions lasting hours or days.
SWE-Agent — Code & Repository Agents
Yang, Jimenez, Wettig et al. (Princeton)
SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering
Introduces the Agent-Computer Interface (ACI) — a purpose-built interface for agents to interact with code repositories. The ACI exposes file editing, bash execution, and search in a format optimised for LLM consumption. Achieves 12.5% on SWE-bench, the strongest open result at publication.
Key insight: The interface between agent and environment matters as much as the agent itself. Raw bash access is too noisy. The ACI translates noisy terminal output into structured, token-efficient observations the model can reason about. This generalises: every tool you give an agent should be designed for LLM consumption.
Generative Agents — Multi-Agent Social Simulation
Park, O'Brien, Cai et al. (Stanford)
Generative Agents: Interactive Simulacra of Human Behavior
25 LLM-powered agents inhabit a virtual town, maintaining memory streams, daily plans, and social relationships. Agents reflect on their memories to form higher-level insights, plan their day, and react to unexpected events. The emergent social behaviours — rumour spreading, relationship formation — were not explicitly programmed.
Key insight: Memory + reflection + planning = emergent behaviour. This paper establishes the three primitives that production multi-agent systems still use today. If your agents need to coordinate over time, this is the architecture to understand.
LangGraph — Stateful Agent Graphs
LangChain Team
LangGraph: Building Stateful, Multi-Actor Applications with LLMs
Models agent execution as a directed graph where nodes are LLM calls or tool actions, and edges encode the control flow — including cycles. State is typed and persisted across steps. Enables complex patterns: parallel tool calls, human-in-the-loop, conditional routing, and agent handoffs.
Key insight: Real agent tasks are not linear. They branch, loop, and parallelize. LangGraph's graph model matches the actual complexity of production tasks in a way that sequential ReAct cannot. This is the current production-grade architecture for non-trivial agents.
Test-Time Compute — The New Scaling Axis
Snell, Lee, Xu et al. (UC Berkeley)
Scaling LLM Test-Time Compute Optimally Beats Scaling Model Parameters
Shows that allocating more compute at inference time — through search, sampling, and verification — can match or beat scaling model parameters. Smaller models with more test-time compute outperform larger models with less on hard reasoning tasks.
Key insight: This reframes agents entirely: an agent that tries multiple approaches, verifies its outputs, and selects the best is fundamentally doing test-time compute scaling. Agent frameworks are not just about automation — they are a compute scaling strategy.
What to Build After This Roadmap
Production agent architecture today (2025):
Task → Planning (decompose into subtasks)
→ Parallel tool execution (LangGraph)
→ Memory management (MemGPT-style)
→ Verification (test-time compute)
→ Human-in-the-loop checkpoints
Related