Evaluating Stochasticity in Deep Research Agents
| Authors | Haotian Zhai et al. |
| Year | 2026 |
| Field | AI / Agents |
| arXiv | 2602.23271 |
| Download | |
| Categories | cs.AI |
Abstract
Deep Research Agents (DRAs) are promising agentic systems that gather and synthesize information to support research across domains such as financial decision-making, medical analysis, and scientific discovery. Despite recent improvements in research quality (e.g., outcome accuracy when ground truth is available), DRA system design often overlooks a critical barrier to real-world deployment: stochasticity. Under identical queries, repeated executions of DRAs can exhibit substantial variability in terms of research outcome, findings, and citations. In this paper, we formalize the study of stochasticity in DRAs by modeling them as information acquisition Markov Decision Processes. We introduce an evaluation framework that quantifies variance in the system and identify three sources of it: information acquisition, information compression, and inference. Through controlled experiments, we investigate how stochasticity from these modules across different decision steps influences the variance of DRA outputs. Our results show that reducing stochasticity can improve research output quality, with inference and early-stage stochasticity contributing the most to DRA output variance. Based on these findings, we propose strategies for mitigating stochasticity while maintaining output quality via structured output and ensemble-based query generation. Our experiments on DeepSearchQA show that our proposed mitigation methods reduce average stochasticity by 22% while maintaining high research quality.
Engineering Breakdown
Plain English
This paper addresses a critical but overlooked problem in Deep Research Agents (DRAs)—systems that autonomously gather and synthesize information for tasks like financial analysis and medical research. The authors found that identical queries to the same DRA produce substantially different outputs across repeated executions, creating a barrier to real-world deployment where consistency is essential. They formalize this stochasticity problem by modeling DRAs as information acquisition Markov Decision Processes and introduce an evaluation framework to quantify variance across system components. The work identifies three specific sources of variability, providing engineers with concrete diagnostics and potential mitigations for building more reliable agentic systems.
Core Technical Contribution
The paper's core novelty is the formalization of stochasticity in agentic systems through an MDP lens, treating information acquisition as a sequential decision problem where randomness can be measured and decomposed. Rather than treating DRA outputs as deterministic or ignoring variance, the authors developed a structured evaluation framework that isolates and quantifies which components of the research pipeline contribute most to output variability. This is fundamentally different from prior work that focuses on accuracy or quality metrics in isolation—it explicitly models how stochastic choices (in search strategies, document selection, or synthesis) cascade through the system. The identification of three distinct variance sources is novel and actionable, enabling targeted interventions rather than blanket approaches.
How It Works
The technical approach models a DRA as an MDP where the agent sequentially acquires information (searches, reads documents, synthesizes findings) with the goal of producing a research output. At each step, the agent's policy—which documents to retrieve, how to synthesize information, which citations to include—introduces stochasticity through sampling or beam search approximations. The evaluation framework runs the same query multiple times and measures variance across output dimensions: factual findings, cited sources, and overall conclusions. By decomposing the system's execution trace, the authors attribute variance to specific components: retrieval stochasticity (randomness in which documents are fetched), synthesis stochasticity (randomness in how information is combined into text), and decision stochasticity (randomness in which research directions the agent explores). The framework then quantifies each source's contribution to overall output variance, creating a diagnostic that pinpoints where determinism or consistency constraints should be applied.
Production Impact
For teams deploying DRAs in high-stakes domains—medical literature review, financial due diligence, scientific discovery—this framework provides a concrete way to measure and reduce unreliability before production. Engineers can use the variance decomposition to decide which components need deterministic behavior (e.g., freezing retrieval rankings) versus which can tolerate stochasticity (e.g., synthesis phrasing). The MDP formulation suggests concrete interventions: you could cache retrieval results, use temperature=0 for LLM synthesis, or enforce deterministic beam search instead of sampling. Adoption requires adding instrumentation to log execution traces and running multiple passes over production queries, increasing compute cost by 2-10x depending on required confidence intervals, but this overhead is negligible compared to the cost of unreliable research outputs in regulatory or clinical contexts. The framework integrates naturally into existing CI/CD pipelines—treating stochasticity as a measurable quality dimension alongside accuracy and latency.
Limitations and When Not to Use This
The paper's scope is limited to the DRA abstraction and may not generalize cleanly to other agentic systems with different decision structures or reward models. The formalization assumes access to and ability to instrument the full execution trace, which is difficult for closed-box systems or LLM APIs where intermediate steps are opaque. The work identifies variance sources but doesn't provide algorithmic solutions for eliminating them in all cases—sometimes the source of stochasticity is fundamental to the agent's performance (e.g., exploration is necessary for finding better information). The evaluation framework also assumes you have ground truth or at least a way to measure output equivalence, which is challenging for open-ended research tasks where multiple valid outputs exist. Follow-up work needs to address how to optimize for both quality and consistency—reducing stochasticity might hurt accuracy if it forces the agent into locally optimal but suboptimal search paths.
Research Context
This work builds on the recent trend of large language model-based agentic systems, extending quality evaluation beyond traditional metrics like BLEU or accuracy to include reliability and reproducibility. It connects to research on uncertainty quantification in neural networks and decision-theoretic approaches to agent design, translating those ideas into the practical setting of information-seeking agents. The paper advances the broader AI safety and reliability agenda by highlighting that modern agentic systems can fail not just by being wrong, but by being inconsistently right—an underexplored failure mode in deployment. It opens a new research direction in characterizing and controlling variance in complex reasoning pipelines, potentially influencing how future agent benchmarks (like AI2's ScienceBench or others for agentic evaluation) measure robustness alongside accuracy.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::