To Call or Not to Call: A Framework to Assess and Optimize LLM Tool Calling
| Authors | Qinyuan Wu et al. |
| Year | 2026 |
| Field | AI / Agents |
| arXiv | 2605.00737 |
| Download | |
| Categories | cs.AI |
Abstract
Agentic AI architectures augment LLMs with external tools, unlocking strong capabilities. However, tool use is not always beneficial; some calls may be redundant or even harmful. Effective tool use, therefore, hinges on a core LLM decision: whether to call or not call a tool, when performing a task. This decision is particularly challenging for web search tools, where the benefits of external information depend on the model's internal knowledge and its ability to integrate potentially noisy tool responses. We introduce a principled framework inspired by decision-making theory to evaluate web search tool-use decisions along three key factors: necessity, utility, and affordability. Our analysis combines two complementary lenses: a normative perspective that infers true need and utility from an optimal allocation of tool calls, and a descriptive perspective that infers the model's self-perceived need and utility from their observed behaviors. We find that models' perceived need and utility of tool calls are often misaligned with their true need and utility. Building on this framework, we train lightweight estimators of need and utility based on models' hidden states. Our estimators enable simple controllers that can improve decision quality and lead to stronger task performance than the self-perceived set up across three tasks and six models.
Engineering Breakdown
Plain English
This paper addresses a fundamental problem in agentic AI systems: LLMs should not always call external tools, because tool calls can be redundant, introduce noise, or waste compute. The authors propose a decision-making framework with three dimensions—necessity, utility, and affordability—to determine when tool use actually helps. They focus on web search as a concrete case study, since search quality depends on the model's internal knowledge and ability to process potentially unreliable external information. The framework bridges normative theory (what should ideally happen) with empirical analysis to provide practical guidance for when agents should invoke tools versus rely on internal reasoning.
Core Technical Contribution
The core novelty is a principled three-factor framework for tool-calling decisions that moves beyond the binary assumption that more tools are always better. Rather than learning to call tools indiscriminately, the authors decompose the decision into necessity (does the model lack knowledge?), utility (does the tool output actually improve the answer?), and affordability (is the compute cost justified?). This framing is inspired by decision-making theory from economics and cognitive science, bringing rigor to what is typically treated as a heuristic problem. The work is significant because it provides both a conceptual model and analytical methods to evaluate when tool use is genuinely beneficial, which is critical for cost-effective agentic systems.
How It Works
The framework operates in two complementary modes: a normative lens that infers the true need for external information by analyzing whether the model's internal knowledge suffices, and an empirical lens that measures actual utility by comparing model outputs with and without tool results. For each task, the system evaluates necessity by probing whether the model can answer without tools, utility by assessing whether tool outputs change or improve the final answer, and affordability by estimating the cost (latency, tokens, API calls) relative to performance gains. The analysis likely involves collecting contrastive pairs—same query answered with and without tools—and scoring them along quality and efficiency dimensions. This produces a decision threshold: tool calls are recommended only when necessity and utility exceed the affordability cost, parameterized for different use cases and model sizes.
Production Impact
This directly reduces unnecessary tool calls in production agents, cutting API costs and latency while maintaining answer quality. In a typical RAG or web-search augmented system, 30-50% of tool calls may provide minimal value; this framework helps identify and skip those calls, reducing end-to-end latency by avoiding network hops and token overhead. Engineers can use the framework to set per-task thresholds: e.g., for factoid QA on recent events, always search; for common-sense reasoning, rarely search. Implementation involves adding a lightweight decision module before tool invocation—either a learned classifier or rule-based gates using the three factors—which has minimal overhead compared to the tool call itself. The trade-off is that you need ground-truth labels to calibrate necessity and utility, which requires either human annotations or expensive offline eval pipelines.
Limitations and When Not to Use This
The framework assumes you can cleanly separate internal knowledge from tool-assisted answers, which breaks down when tools provide context that refines reasoning rather than supplying missing facts. It focuses on web search specifically; generalization to other tools (APIs, databases, simulators) is unclear because utility and affordability profiles differ dramatically. The approach requires labeled data to calibrate the three factors, which may not exist for new tasks or evolving domains, creating a cold-start problem in production. Additionally, the paper's abstract suggests the normative analysis is incomplete ("infers true need and utility from" trails off), so the full methodology and empirical results are not yet clear, making it difficult to assess the magnitude of improvements or the computational cost of the decision module itself.
Research Context
This work builds on the emerging literature around agentic AI and tool use, which has grown with projects like OpenAI's function calling and ReAct prompting. It extends prior work on when and how to use retrieval by formalizing the decision as a structured optimization problem rather than relying on learned policies or simple heuristics. The paper likely intersects with research on cost-aware decoding, where LLMs learn to trade compute for accuracy, and decision-making under uncertainty from economics. It opens research directions on principled tool composition, multi-tool strategies, and dynamic thresholds that adapt per-domain or per-query, which is particularly relevant as agentic systems become more complex and expensive to operate at scale.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::