Agents Explore but Agents Ignore: LLMs Lack Environmental Curiosity
| Authors | Leon Engländer et al. |
| Year | 2026 |
| HF Upvotes | 7 |
| arXiv | 2604.17609 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
LLM-based agents are assumed to integrate environmental observations into their reasoning: discovering highly relevant but unexpected information should naturally lead to a model exploiting its own discoveries. We show that this assumption is false for current LLM-based agents, which struggle to reflect or react to unexpected information. Across three benchmarks (Terminal-Bench, SWE-Bench, AppWorld), we inject complete task solutions into the agent environments to deliberately expose a task's solution to a model. While agents discover these solutions on Terminal-Bench in 79-81% of runs, they interact, or exploit, them in only 37-50% of cases. This gap is starkest in AppWorld: agents see documentation stating that a command "returns the complete solution to this task" in over 90% of attempts but exploit this in fewer than 7% of trials. We show that agents lack what we call environmental curiosity: the capability to recognize and investigate unexpected but relevant observations in response to environmental stimuli. We identify three main factors influencing environmental curiosity: available tools in the agent scaffold, test-time compute, and training data distribution. Our findings identify configurations that maximize curiosity also achieve the best performance on the unmodified benchmarks. Yet even jointly optimized agents still ignore discovered solutions in the majority of trials: current agents use the environment to fetch expected information, but not to revise their strategy or maximally exploit useful stimuli.
Engineering Breakdown
Plain English
This paper reveals a critical failure mode in LLM-based agents: they struggle to exploit information they discover in their environments, even when that information directly solves their task. The researchers tested this across three benchmarks (Terminal-Bench, SWE-Bench, AppWorld) by deliberately injecting complete task solutions into agent environments. Agents discovered these solutions 79-81% of the time on Terminal-Bench, but only 37-50% of the time actually used them to complete the task. In AppWorld, the gap was most severe—agents encountered documentation explicitly stating "returns the complete solution to this task" over 90% of the time but failed to leverage it, revealing that current LLM agents lack genuine environmental curiosity and adaptive reasoning.
Core Technical Contribution
The paper's core contribution is empirically demonstrating that LLM-based agents have a fundamental architectural limitation: they observe their environment but fail to meaningfully integrate unexpected discoveries into their decision-making process. Rather than proposing a new method, the authors provide a diagnostic framework using deliberately-planted solutions as a probe to measure agent adaptability—exposing the gap between discovery and exploitation. This challenges a foundational assumption in agent design: that if an LLM reads something relevant, it will naturally use it. The work is novel because it quantifies this failure across multiple domains and shows it's not a training or prompting problem, but a deeper limitation in how LLM-based agents reason about environmental context.
How It Works
The experimental setup involves three stages: (1) agents operate in controlled environments where complete task solutions are injected as discoverable information—either embedded in command output, documentation, or system responses; (2) researchers measure discovery rates by checking whether agents' action logs or reasoning traces reference the injected solution; (3) researchers measure exploitation by determining whether agents subsequently use the discovered solution to complete the task. The key mechanism is comparison: a solution's presence/absence is randomized or varied, allowing clean measurement of whether discovery causally leads to usage. On Terminal-Bench (command-line tasks), agents might discover that running a specific command outputs the answer directly; the gap occurs when agents see this output but then continue attempting other approaches instead of recognizing they've solved the problem. On AppWorld (interactive application environment), documentation explicitly states "this function solves the task," yet agents ignore it or forget it within a few subsequent actions, suggesting working memory or attention limitations within the agent's reasoning loop.
Production Impact
For teams building production LLM agents, this finding means that adding rich environmental observations and knowledge bases alone will not improve task completion—agents need architectural changes to properly reflect on and integrate discovered information. Current deployments (code assistants, automation agents, autonomous researchers) likely have hidden inefficiencies where agents pass over valid solutions they technically encountered. To address this in production, engineers would need to implement explicit verification loops: after agents take actions and receive observations, inject a reflection step that forces the agent to extract and rank high-value information before proceeding. This adds latency (potentially 10-30% overhead depending on model size and reasoning depth) and requires careful prompt engineering to avoid hallucination, but could significantly improve task completion rates without retraining. Teams should also implement observability to detect when agents ignore relevant environmental information—flagging these cases would reveal whether the issue is discovery, interpretation, or decision-making.
Limitations and When Not to Use This
The paper doesn't propose solutions, only diagnoses the problem—it leaves open the question of whether this limitation stems from attention mechanisms, context window management, or deeper architectural misalignments in how LLMs process sequential decision-making. The three benchmarks tested (Terminal-Bench, SWE-Bench, AppWorld) are all relatively structured environments with clean state spaces; the findings may not generalize to noisy, real-world settings where irrelevant information heavily outnumbers relevant discoveries. The injection of solutions is artificial and oracle-like; it's unclear whether agents would behave differently if solutions were embedded more naturally or required synthesis from multiple pieces of information. The paper also doesn't isolate whether the failure is in the LLM's reasoning capability, the agent framework's memory management, or the prompting strategy—this leaves engineers uncertain about which intervention (fine-tuning, architecture redesign, or better prompts) would actually fix the problem in practice.
Research Context
This work builds on the emerging critique of LLM-based agents as reasoning systems, extending recent findings that LLMs struggle with long-horizon planning, credit assignment, and closed-loop feedback. It relates to prior work on agent interpretability and failure analysis (e.g., studies showing agents ignore crucial context or repeat failed actions) but provides a systematic empirical framework for measuring discovery-vs.-exploitation gaps. The paper contributes to the broader discussion around whether LLMs genuinely reason over observations or merely pattern-match, influencing how the community approaches agent design—suggesting that naïve scaling of context window and information density won't solve core reasoning problems. The benchmarks used (Terminal-Bench, SWE-Bench, AppWorld) represent a progression from simple to complex task domains, establishing a testing methodology that future work can use to evaluate improvements in agent adaptability and environmental awareness.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::