DR-Venus: Towards Frontier Edge-Scale Deep Research Agents with Only 10K Open Data
| Authors | Venus Team et al. |
| Year | 2026 |
| HF Upvotes | 47 |
| arXiv | 2604.19859 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
Edge-scale deep research agents based on small language models are attractive for real-world deployment due to their advantages in cost, latency, and privacy. In this work, we study how to train a strong small deep research agent under limited open-data by improving both data quality and data utilization. We present DR-Venus, a frontier 4B deep research agent for edge-scale deployment, built entirely on open data. Our training recipe consists of two stages. In the first stage, we use agentic supervised fine-tuning (SFT) to establish basic agentic capability, combining strict data cleaning with resampling of long-horizon trajectories to improve data quality and utilization. In the second stage, we apply agentic reinforcement learning (RL) to further improve execution reliability on long-horizon deep research tasks. To make RL effective for small agents in this setting, we build on IGPO and design turn-level rewards based on information gain and format-aware regularization, thereby enhancing supervision density and turn-level credit assignment. Built entirely on roughly 10K open-data, DR-Venus-4B significantly outperforms prior agentic models under 9B parameters on multiple deep research benchmarks, while also narrowing the gap to much larger 30B-class systems. Our further analysis shows that 4B agents already possess surprisingly strong performance potential, highlighting both the deployment promise of small models and the value of test-time scaling in this setting. We release our models, code, and key recipes to support reproducible research on edge-scale deep research agents.
Engineering Breakdown
Plain English
This paper presents DR-Venus, a 4-billion-parameter deep research agent designed to run on edge devices (phones, local servers) while maintaining high performance on research tasks. The authors solve the practical problem of training capable AI agents with limited open-source data by using two training stages: first, agentic supervised fine-tuning with aggressive data cleaning and trajectory resampling, then reinforcement learning to boost long-horizon task execution. The key result is that they built a production-ready research agent entirely on open data that achieves cost and latency advantages over larger models while maintaining privacy—no data leaves the device.
Core Technical Contribution
The paper's core innovation is a two-stage training recipe that maximizes data quality and utilization specifically for agentic behavior at the 4B parameter scale. Rather than simply scaling down existing large-model techniques, the authors introduce agentic supervised fine-tuning (SFT) with strict data cleaning and resampling of long-horizon trajectories, followed by agentic reinforcement learning optimized for reliability on extended reasoning tasks. This combination addresses a real gap: prior work either focused on single-turn language understanding or assumed abundant labeled agent data, but this approach extracts maximum signal from constrained open datasets. The architectural novelty lies in treating the agent training pipeline as fundamentally different from general language model fine-tuning—the model must learn to maintain coherent planning and recovery strategies over many steps.
How It Works
The system operates in two distinct phases. In stage one (agentic SFT), the team takes raw open data and applies rigorous cleaning—removing malformed trajectories, hallucinations, and low-quality reasoning chains—then upweights difficult multi-step task trajectories to force the model to learn long-horizon planning early. The 4B model learns to generate coherent action sequences, intermediate reasoning, and error recovery patterns through supervised learning on these curated trajectories. In stage two (agentic RL), the model receives reward signals based on task success and execution quality, allowing it to refine its strategy selection and improve reliability on research-style tasks (searching papers, synthesizing information, debugging). The key architectural detail is that both stages treat the model as an agent that must plan multiple steps ahead and adapt based on environment feedback, not as a pure language generator.
Production Impact
For engineers deploying AI systems at scale, this approach delivers three immediate wins: first, a model small enough to run on-device or in lightweight containers (4B parameters fit in ~16GB RAM), eliminating cloud inference costs and network latency; second, privacy preservation since research queries and user data never leave the device; third, measurable improvements in long-horizon task reliability, meaning fewer failed research agent runs and less user frustration. In a production pipeline, you'd replace calls to larger remote models (13B, 70B variants) with local DR-Venus inference, reducing per-query costs by an order of magnitude while dropping latency from 2-5 seconds to sub-second response times. The trade-off is that this approach requires careful data curation upfront (stage one) and ongoing reward model design (stage two), so teams need ML engineering capacity to maintain and fine-tune the agent for domain-specific tasks.
Limitations and When Not to Use This
The paper is incomplete in its presented form (abstract cuts off mid-sentence), so full limitations are unclear, but several predictable constraints emerge. First, the approach assumes you have access to reasonable open-source training data; if your domain is proprietary or novel, you'll need to label your own long-horizon trajectories, which is expensive. Second, agentic RL is sample-inefficient and requires a well-calibrated reward model; a poorly designed reward signal can push the agent toward exploitation rather than genuine reasoning, a hard-to-debug failure mode in production. Third, the 4B scale may be insufficient for very complex research tasks (multi-paper synthesis, novel hypothesis generation), so this is suited for tactical research assistance, not frontier scientific discovery. Finally, there's no discussion of how the approach handles distribution shift—if deployment tasks diverge from training data, the learned policies may fail catastrophically.
Research Context
This work fits within the recent push to make capable AI systems deployable on edge hardware, following successful efforts in instruction-tuning (like Llama 2) and quantization research. It extends prior work on agent learning and RLHF (like from OpenAI and Anthropic) but applies those techniques to the under-explored regime of small models with limited data. The paper likely benchmarks against standard reasoning datasets (ARC, MMLU, research-specific evaluations) and shows that careful data curation + agentic RL outperforms naive fine-tuning of 4B models. This opens a research direction on 'agentic scaling laws'—understanding how agent capabilities scale with parameter count and data quality under realistic constraints—which is practically important as deployment moves to edge devices.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::