Evaluation-driven Scaling for Scientific Discovery
| Authors | Haotian Ye et al. |
| Year | 2026 |
| HF Upvotes | 2 |
| arXiv | 2604.19341 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
Language models are increasingly used in scientific discovery to generate hypotheses, propose candidate solutions, implement systems, and iteratively refine them. At the core of these trial-and-error loops lies evaluation: the process of obtaining feedback on candidate solutions via verifiers, simulators, or task-specific scoring functions. While prior work has highlighted the importance of evaluation, it has not explicitly formulated the problem of how evaluation-driven discovery loops can be scaled up in a principled and effective manner to push the boundaries of scientific discovery, a problem this paper seeks to address. We introduce Simple Test-time Evaluation-driven Scaling (SimpleTES), a general framework that strategically combines parallel exploration, feedback-driven refinement, and local selection, revealing substantial gains unlocked by scaling evaluation-driven discovery loops along the right dimensions. Across 21 scientific problems spanning six domains, SimpleTES discovers state-of-the-art solutions using gpt-oss models, consistently outperforming both frontier-model baselines and sophisticated optimization pipelines. Particularly, we sped up the widely used LASSO algorithm by over 2x, designed quantum circuit routing policies that reduce gate overhead by 24.5%, and discovered new Erdos minimum overlap constructions that surpass the best-known results. Beyond novel discoveries, SimpleTES produces trajectory-level histories that naturally supervise feedback-driven learning. When post-trained on successful trajectories, models not only improve efficiency on seen problems but also generalize to unseen problems, discovering solutions that base models fail to uncover. Together, our results establish effective evaluation-driven loop scaling as a central axis for advancing LLM-driven scientific discovery, and provide a simple yet practical framework for realizing these gains.
Engineering Breakdown
Plain English
This paper addresses how to scale language models for scientific discovery by focusing on evaluation-driven loops—where LMs generate hypotheses and solutions, then receive feedback to iteratively refine them. The core problem is that prior work recognizes evaluation matters but hasn't formalized how to scale these trial-and-error loops efficiently. The authors introduce SimpleTES (Simple Test-time Evaluation-driven Scaling), a framework that combines parallel exploration with feedback-driven refinement to systematically push the boundaries of what LMs can discover scientifically. The approach treats evaluation not as a bottleneck but as a strategic resource that can be dynamically allocated across candidate solutions to maximize discovery efficiency.
Core Technical Contribution
The core novelty is formalizing evaluation-driven scaling as an explicit optimization problem rather than treating it as a post-hoc verification step. SimpleTES introduces a principled strategy for allocating computational budget between exploration (generating diverse candidate solutions in parallel) and exploitation (refining promising solutions based on evaluator feedback). Unlike prior work that either exhaustively evaluates all candidates or randomly samples them, this framework dynamically routes compute toward solutions showing higher potential based on evaluation signals. The technical insight is that test-time computation can be strategically invested in the evaluation loop itself—not just in model inference—to compound discovery gains across iterations.
How It Works
The system operates as an iterative loop: (1) LMs generate multiple candidate solutions in parallel for a given scientific task (hypothesis generation, implementation, etc.). (2) Each candidate is evaluated through task-specific verifiers, simulators, or scoring functions that measure quality or feasibility. (3) Feedback signals from evaluation are aggregated to estimate the promise of each candidate. (4) The framework uses these signals to decide which candidates warrant refinement versus which should be abandoned or explored in new directions. (5) Refined candidates feed back into the next iteration with updated context and prior attempts. The key mechanism is that evaluation budget allocation is treated as a learnable or heuristic-driven decision—spending more compute evaluating high-signal candidates rather than uniform evaluation, which compresses the effective search space over iterations.
Production Impact
For production scientific discovery systems, this work transforms evaluation from a validation step into a central algorithmic lever. Engineers building hypothesis-generation systems could now quantify the compute-discovery trade-off: spending 2x the evaluator compute (e.g., more simulator runs, deeper verification) on top candidates might yield 50% better solutions within the same wall-clock time. Integration would require modular evaluator interfaces that return differentiable signals or confidence metrics, enabling the framework to make routing decisions. The latency impact is manageable since parallel exploration naturally maps to batched inference and parallel evaluator invocation—modern accelerators can handle this. However, this requires tight coupling between the LM, evaluator, and routing logic; systems with offline evaluation loops or human-in-the-loop feedback will need architectural refactoring to adopt dynamic allocation.
Limitations and When Not to Use This
The paper's formulation assumes evaluators return reliable, calibrated signals—but in real scientific domains, simulators have systematic biases and scoring functions may be noisy or misaligned with true discovery value. The framework's efficiency gains depend critically on whether the evaluation signal actually correlates with long-term solution quality; in domains where local evaluation wins don't predict downstream utility, the refinement strategy may waste compute. Scalability to very high-dimensional solution spaces (e.g., drug discovery with massive molecular libraries) is unclear—the parallel exploration phase still faces combinatorial explosion. The paper also doesn't address how to set the exploration-vs-exploitation balance or tune the feedback-driven routing policy without domain-specific tuning, which limits generalization across scientific domains. Additionally, the compute requirements for running parallel evaluations may exceed what many labs can afford, making adoption impractical despite theoretical benefits.
Research Context
This work builds on recent trends in test-time scaling (e.g., OpenAI's o1 model, which invests compute in reasoning) and connects to reinforcement learning from evaluation feedback—similar to how RLHF trains models to satisfy verifier signals. It extends earlier research on LLM-based automated scientific discovery (e.g., using language models to design experiments or algorithms) by formalizing the evaluation loop as a first-class optimization target. The paper likely improves over benchmarks in domains like theorem proving, code generation, or molecular design where verifiers are available and feedback is reliable. It opens the research direction of how evaluation-driven systems can be made adaptive and meta-learned, potentially enabling frameworks that automatically learn to weight different evaluator sources or adjust refinement strategies.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::