LLMs Improving LLMs: Agentic Discovery for Test-Time Scaling
:::info Stub — Full Engineering Breakdown Coming This paper has a linked code implementation and was featured on Hugging Face Papers with 36 upvotes. A full breakdown with production viability rating, implementation notes, and honest limitations is being written. Subscribe to AI Letters → :::
| Authors | Tong Zheng et al. |
| Year | 2026 |
| HF Upvotes | 36 |
| arXiv | 2605.08083 |
| Download | |
| Code | https://github.com/zhengkid/AutoTTS |
Abstract
Test-time scaling (TTS) has become an effective approach for improving large language model performance by allocating additional computation during inference. However, existing TTS strategies are largely hand-crafted: researchers manually design reasoning patterns and tune heuristics by intuition, leaving much of the computation-allocation space unexplored. We propose an environment-driven framework, AutoTTS, that changes what researchers design: from individual TTS heuristics to environments where TTS strategies can be discovered automatically. The key to AutoTTS lies in environment construction: the discovery environment must make the control space tractable and provide cheap, frequent feedback for TTS search. As a concrete instantiation, we formulate width--depth TTS as controller synthesis over pre-collected reasoning trajectories and probe signals, where controllers decide when to branch, continue, probe, prune, or stop and can be evaluated cheaply without repeated LLM calls. We further introduce beta parameterization to make the search tractable and fine-grained execution trace feedback to improve discovery efficiency by helping the agent diagnose why a TTS program fails. Experiments on mathematical reasoning benchmarks show that the discovered strategies improve the overall accuracy--cost tradeoff over strong manually designed baselines. The discovered strategies generalize to held-out benchmarks and model scales, while the entire discovery costs only $39.9 and 160 minutes. Our data, and code will be open-source at https://github.com/zhengkid/AutoTTS.
Engineering Breakdown
Plain English
This paper proposes AutoTTS, a framework that automatically discovers test-time scaling strategies for LLMs instead of relying on hand-crafted heuristics. Rather than manually designing reasoning patterns, the authors built an environment where an agent can search for optimal compute allocation strategies during inference, formulating the problem as controller synthesis across width-depth tradeoffs.
Key Engineering Insight
The core innovation is shifting from designing individual TTS heuristics to designing discoverable environments—by making the control space tractable and enabling cheap, frequent feedback loops, you can let agents autonomously find better compute-allocation strategies than humans would manually tune.
Why It Matters for Engineers
Test-time compute is one of the last levers for improving LLM quality without retraining. Most production systems use intuition-based strategies (like chain-of-thought or majority voting). AutoTTS could unlock better performance-per-inference-dollar by discovering non-obvious allocation patterns, directly impacting model serving costs and quality tradeoffs at scale.
Research Context
Test-time scaling is a well-studied area, but existing approaches rely on manual heuristic design and empirical tuning. This paper advances the field by automating strategy discovery itself—treating TTS as a synthesis problem rather than an engineering problem. This opens a path toward continuously improving inference without architectural changes or retraining.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::