AI Letters #06 · Tool Use Deep Dive

The Evolution of Tool Use in AI Systems

From rule-based expert systems to the Model Context Protocol — click any card to expand

16K+
APIs benchmarked in ToolBench (2023)
47%
Reduction in tool hallucination (Gorilla, 2024)
2,400+
MCP-compatible tools by end of 2025
90.2%
Best model tool calling accuracy (BFCL 2025)
1980s Expert Systems with Rule-Based Tools Rules Era

Expert systems like MYCIN and DENDRAL used hand-coded rule engines as their "tools." Each rule was a conditional action: IF symptom_X AND symptom_Y THEN prescribe_Z. The tools were domain-specific, brittle, and required manual updates for every new case. There was no general-purpose tool interface — each system was its own closed world.

Engineering Impact
Proved that structured action schemas work. Every modern tool schema (JSON function definitions) is a direct descendant of expert system rule formalisms. The key insight survived: you need a formal representation of what an action does before an inference engine can use it.
1997 Deep Blue: Domain-Specific Tool Integration Rules Era

IBM's Deep Blue defeated Garry Kasparov by deeply integrating hardware-accelerated chess evaluation as a "tool" for its search algorithm. The system called its evaluation function millions of times per second. The lesson: tight integration between a reasoning system and specialized tools can achieve superhuman performance in bounded domains. The boundary was the key word.

Engineering Impact
Domain-specific tool integration outperforms general-purpose reasoning in closed domains. The failure mode: tools that only work in one domain require complete rebuilds for new problems. LLM-based agents solve this by using the same tool protocol across arbitrary domains.
2016 OpenAI Gym: Standardized Environment Interfaces API Era

OpenAI Gym standardized the environment interface for reinforcement learning: env.step(action) returns (observation, reward, done, info). This created a common API across hundreds of environments — the same agent code could train on Atari, MuJoCo, or a custom environment. It was the first widely-adopted standard for "agent takes action, environment returns observation."

Engineering Impact
The Gym interface directly prefigures MCP. Standardize the action/observation contract, and any agent works with any environment. MCP generalizes this concept from RL environments to LLM tools — the same structural insight, broader application.
2022 ReAct Paper: Tools + Reasoning Formalized Agent Era

Yao et al. published "ReAct: Synergizing Reasoning and Acting in Language Models" (arXiv:2210.03629). The paper formalized the THOUGHT/ACTION/OBSERVATION loop and showed empirically that interleaving reasoning with tool use dramatically outperformed either alone on HotpotQA (35.1% vs 25.7% for action-only) and AlfWorld (71% vs 45%). This was the moment the academic community had a rigorous framework for what practitioners were already doing ad-hoc.

Engineering Impact
ReAct gave engineers a vocabulary and a pattern. Every modern agentic framework — LangChain, AutoGen, CrewAI, LlamaIndex Agents — is an implementation of the ReAct loop. If you understand the THOUGHT/ACTION/OBSERVATION cycle, you understand all of them.
Jan 2023 ChatGPT Plugins: First Mainstream Tool Platform Agent Era

OpenAI launched the ChatGPT Plugins beta — the first mainstream product that let end users connect LLMs to external tools via a standardized API spec. Plugins used OpenAPI schemas to describe tool capabilities, which ChatGPT would read and use for tool selection. Within weeks, hundreds of third-party tools (Wolfram, Zapier, Expedia) had plugin integrations.

Engineering Impact
Proved product-market fit for LLM + tools at scale. Also revealed the first large-scale failure modes: tools that returned too much data, tool descriptions that caused confusion, and safety issues with unconstrained external tool access. The lessons directly shaped function calling API design and, later, MCP.
Mar 2023 GPT-4 Function Calling API Released Agent Era

OpenAI released the function calling API for GPT-4, providing a structured way to define tools via JSON schema and receive structured JSON back from the model when it wanted to call a tool. This replaced ad-hoc prompting tricks ("output JSON like this...") with a first-class protocol. The model was explicitly fine-tuned to produce valid function calls, not just text that looks like function calls.

Engineering Impact
Function calling API reliability jumped to ~85%+ vs ~60% with prompt-based tool selection. The structured output also made parsing trivial — no more regex hacks to extract tool calls from model text. Every major LLM provider subsequently released their own function calling API using the same JSON schema pattern.
Aug 2023 ToolBench Paper: 16,000+ Tool Benchmark Agent Era

Shi et al. (2023) released ToolBench: a benchmark of 16,464 real-world APIs from RapidAPI, covering 49 categories. The paper showed that GPT-4 achieved only 49% success on complex multi-tool tasks out-of-the-box, and that description quality was the single biggest predictor of tool selection accuracy. Fine-tuning on ToolBench raised open-source model performance by 28 percentage points.

Engineering Impact
First rigorous benchmark for tool-calling agents. Confirmed that description quality is the primary lever, not model size or prompt length alone. Also introduced the concept of tool retrieval — when you have 16K tools, you can't fit all schemas in the context window; you need to retrieve the relevant subset.
Jan 2024 Gorilla Paper: Tool Hallucination Research Agent Era

Zhuang et al. (2024) published Gorilla, a model fine-tuned specifically on API documentation. The key finding: LLMs hallucinate tool arguments significantly — especially when APIs change after the training cutoff. Gorilla introduced retrieval-augmented tool use: fetch the current API doc at inference time, not from training data. This reduced hallucination rates by 47% compared to GPT-4 on the same benchmark.

Engineering Impact
Hallucination in tool calls is a retrieval problem, not just a model problem. Always fetch current API documentation, schemas, and constraints at runtime rather than relying on training data. This directly motivated the "Resources" primitive in MCP: the server can serve current documentation that the agent retrieves before calling tools.
Nov 2024 Anthropic MCP Protocol Released MCP Era

Anthropic open-sourced the Model Context Protocol: a JSON-RPC 2.0-based standard for connecting LLMs to tools, resources, and prompts. MCP defines three primitives: Tools (executable functions with schemas), Resources (files and data the model can read), and Prompts (reusable templates). Transport layer supports stdio (local) and HTTP/SSE (remote). The spec was immediately open-sourced at modelcontextprotocol.io.

Engineering Impact
MCP solves the N-squared integration problem: without it, N agents times M tool providers = N*M custom integrations. With MCP, any MCP client works with any MCP server. The protocol is transport-agnostic and language-agnostic — servers exist in Python, TypeScript, Rust, Go. This is the USB-C moment for AI tooling.
2025 MCP Adoption: VS Code, Claude Desktop, Major IDEs MCP Era

By mid-2025, MCP had been adopted by VS Code (GitHub Copilot), Claude Desktop, Cursor, Windsurf, Zed, and JetBrains IDEs. Enterprise adopters surpassed 340 companies. The MCP registry had 2,400+ community-built servers covering databases, cloud APIs, dev tools, and SaaS platforms. OpenAI announced MCP support in the ChatGPT API, effectively making it the industry standard.

Engineering Impact
If you build your tools as MCP servers today, they work with every major IDE's AI assistant, every major LLM API, and any future framework that adopts the standard. The engineering ROI of writing an MCP server vs ad-hoc tool integration is now clearly positive at any team size beyond one person.
www.engineersofai.com · AI Letters #06 · Agentic AI A-Z Series