HyperEyes: Dual-Grained Efficiency-Aware Reinforcement Learning for Parallel Multimodal Search Agents
:::info Stub — Full Engineering Breakdown Coming This paper has a linked code implementation and was featured on Hugging Face Papers with 6 upvotes. A full breakdown with production viability rating, implementation notes, and honest limitations is being written. Subscribe to AI Letters → :::
| Authors | Guankai Li et al. |
| Year | 2026 |
| HF Upvotes | 6 |
| arXiv | 2605.07177 |
| Download | |
| Code | https://github.com/DeepExperience/HyperEyes |
Abstract
Existing multimodal search agents process target entities sequentially, issuing one tool call per entity and accumulating redundant interaction rounds whenever a query decomposes into independent sub-retrievals. We argue that effective multimodal agents should search wider rather than longer: dispatching multiple grounded queries concurrently within a round. To this end, we present HyperEyes, a parallel multimodal search agent that fuses visual grounding and retrieval into a single atomic action, enabling concurrent search across multiple entities while treating inference efficiency as a first-class training objective. HyperEyes is trained in two stages. For cold-start supervision, we develop a Parallel-Amenable Data Synthesis Pipeline covering visual multi-entity and textual multi-constraint queries, curating efficiency-oriented trajectories via Progressive Rejection Sampling. Building on this, our central contribution, a Dual-Grained Efficiency-Aware Reinforcement Learning framework, operates at two levels. At the macro level, we propose TRACE (Tool-use Reference-Adaptive Cost Efficiency), a trajectory-level reward whose reference is monotonically tightened during training to suppress superfluous tool calls without restricting genuine multi-hop search. At the micro level, we adapt On-Policy Distillation to inject dense token-level corrective signals from an external teacher on failed rollouts, mitigating the credit-assignment deficiency of sparse outcome rewards. Since existing benchmarks evaluate accuracy as the sole metric, omitting inference cost, we introduce IMEB, a human-curated benchmark of 300 instances that jointly evaluates search capability and efficiency. Across six benchmarks, HyperEyes-30B surpasses the strongest comparable open-source agent by 9.9% in accuracy with 5.3x fewer tool-call rounds on average.
Engineering Breakdown
Plain English
HyperEyes is a multimodal search agent that processes multiple entities in parallel within a single round, rather than sequentially calling tools one entity at a time. The system combines visual grounding and retrieval into atomic parallel actions and treats inference efficiency as a first-class training objective, using a two-stage training approach with a data synthesis pipeline to enable cold-start supervision.
Key Engineering Insight
The core innovation is treating parallelization as a fundamental architectural constraint during training, not bolting it on afterward. By making efficiency part of the reward signal during RL training, the agent learns to naturally batch independent sub-retrievals instead of developing sequential habits that are hard to retrofit.
Why It Matters for Engineers
For production multimodal systems, query latency scales with sequential depth. If your agent naturally chains tool calls serially (visual grounding → retrieval → visual grounding → retrieval), response time balloons. HyperEyes' parallel execution with efficiency-aware training directly reduces wall-clock time for decomposable queries, improving user experience without requiring post-hoc optimization tricks.
Research Context
Prior multimodal agents treated parallelization as an execution-time problem rather than a training-time design choice, leading to sequentially-biased learned policies. HyperEyes advances beyond this by making the agent learn efficient parallel decomposition patterns from the ground up, enabling a new class of faster, more scalable multimodal retrieval systems that can handle complex multi-entity queries without the latency overhead of sequential tool calling.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::