When Reasoning Models Hurt Behavioral Simulation: A Solver-Sampler Mismatch in Multi-Agent LLM Negotiation
| Authors | Sandro Andric |
| Year | 2026 |
| HF Upvotes | 2 |
| arXiv | 2604.11840 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
Large language models are increasingly used as agents in social, economic, and policy simulations. A common assumption is that stronger reasoning should improve simulation fidelity. We argue that this assumption can fail when the objective is not to solve a strategic problem, but to sample plausible boundedly rational behavior. In such settings, reasoning-enhanced models can become better solvers and worse simulators: they can over-optimize for strategically dominant actions, collapse compromise-oriented terminal behavior, and sometimes exhibit a diversity-without-fidelity pattern in which local variation survives without outcome-level fidelity. We study this solver-sampler mismatch in three multi-agent negotiation environments adapted from earlier simulation work: an ambiguous fragmented-authority trading-limits scenario, an ambiguous unified-opposition trading-limits scenario, and a new-domain grid-curtailment case in emergency electricity management. We compare three reflection conditions, no reflection, bounded reflection, and native reasoning, across two primary model families and then extend the same protocol to direct OpenAI runs with GPT-4.1 and GPT-5.2. Across all three experiments, bounded reflection produces substantially more diverse and compromise-oriented trajectories than either no reflection or native reasoning. In the direct OpenAI extension, GPT-5.2 native ends in authority decisions in 45 of 45 runs across the three experiments, while GPT-5.2 bounded recovers compromise outcomes in every environment. The contribution is not a claim that reasoning is generally harmful. It is a methodological warning: model capability and simulation fidelity are different objectives, and behavioral simulation should qualify models as samplers, not only as solvers.
Engineering Breakdown
Plain English
This paper identifies a critical gap in using large language models as agents in multi-agent simulations: stronger reasoning capabilities can actually degrade simulation fidelity. The authors discovered that enhanced LLMs tend to over-optimize toward strategically dominant actions rather than sampling the bounded-rational, compromise-seeking behaviors that occur in real negotiations. They studied this "solver-sampler mismatch" across three multi-agent negotiation environments and found that reasoning-enhanced models can exhibit a "diversity-without-fidelity" pattern where local behavioral variation exists but fails to match real-world outcomes.
Core Technical Contribution
The paper's core insight is that optimizing for solution quality (solver objective) and sampling plausible human-like behavior (sampler objective) are fundamentally misaligned tasks in simulation contexts. The authors formally articulate why stronger reasoning capacity causes models to collapse toward dominated-strategy equilibria that don't reflect actual negotiator behavior, which violates the implicit assumption that "better reasoning equals better simulation." This is a theoretical contribution that reframes the design goal: simulators need bounded rationality constraints, not unlimited reasoning, to maintain behavioral fidelity. The work introduces measurable failure modes—outcome-level fidelity loss despite surface-level diversity—that can be used to evaluate agent-based simulations.
How It Works
The methodology involves three nested components: (1) multi-agent negotiation environments where agents must reach agreements under incomplete information and conflicting preferences, adapted from prior simulation literature; (2) comparison of LLM agents with varying reasoning capabilities—from base models to reasoning-enhanced variants; (3) measurement frameworks tracking both strategic optimality (solver metrics) and behavioral realism (sampler metrics like match to human negotiation patterns). The key mechanism is that standard LLM generation, when trained on diverse human behavior, produces outputs sampled from a mixed distribution reflecting compromises, risk-aversion, and bounded cognition. When you enhance reasoning through techniques like chain-of-thought or process supervision, the model's sampling distribution shifts toward high-value actions predicted by game-theoretic analysis, losing the fat tails of human behavior. The paper measures this mismatch by comparing terminal outcomes (whether agreements reached match empirical distributions) versus intermediate action diversity (whether agents take varied actions mid-negotiation).
Production Impact
For engineers building agent-based economic or policy simulations, this paper mandates a fundamental architectural choice: explicitly control agent rationality rather than assuming it correlates with model strength. In practice, this means either (1) intentionally using weaker base models for behavioral realism, (2) adding constraints that force exploration or bounded-optimal behavior, or (3) training separate "reasoning" and "behavioral sampling" heads with different objectives. The compute implication is counterintuitive—you may need more inference capacity to run less-capable models across longer rollouts to reach convergence, but you gain simulation validity you'd otherwise lose. For policy simulation pipelines, the trade-off is clear: increased model capability (higher cost, longer latency) reduces outcome fidelity unless you add behavioral constraints, creating a paradoxical need to deliberately handicap reasoning for accuracy.
Limitations and When Not to Use This
The paper's scope is limited to negotiation environments and doesn't establish how broadly the solver-sampler mismatch applies across other simulation domains (cooperation games, auction mechanisms, etc.). The work assumes that empirical human negotiation data exists as a ground truth for fidelity evaluation, which may not hold for novel policy scenarios where historical behavior is sparse or unrepresentative. The paper doesn't provide a principled algorithm for diagnosing when a given simulation should optimize for solving versus sampling, leaving practitioners to manually identify the mismatch case-by-case. Additionally, the "diversity-without-fidelity" pathology is observed empirically but lacks theoretical guarantees about when or why it occurs, limiting predictive power for new architectures.
Research Context
This work sits at the intersection of agent-based modeling (a decades-old simulation paradigm in economics and policy) and modern LLM capabilities, challenging the intuition that scaling reasoning automatically improves simulation accuracy. It builds on prior work in bounded rationality theory and quantal response models, which explicitly model human suboptimality, and connects to recent debates about LLMs as agent simulators (like work on LLM-based population models). The paper opens a research direction around disentangling reasoning and behavioral sampling objectives, with potential applications to synthetic data generation, counterfactual scenario analysis, and policy evaluation. It also implicitly critiques the "bigger and better" narrative in AI by showing that task objectives fundamentally determine whether capability scaling helps or hurts.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::