CoME: Empowering Channel-of-Mobile-Experts with Informative Hybrid-Capabilities Reasoning
| Authors | Yuxuan Liu et al. |
| Year | 2026 |
| Field | NLP |
| arXiv | 2602.24142 |
| Download | |
| Categories | cs.CL, cs.AI |
Abstract
Mobile Agents can autonomously execute user instructions, which requires hybrid-capabilities reasoning, including screen summary, subtask planning, action decision and action function. However, existing agents struggle to achieve both decoupled enhancement and balanced integration of these capabilities. To address these challenges, we propose Channel-of-Mobile-Experts (CoME), a novel agent architecture consisting of four distinct experts, each aligned with a specific reasoning stage, CoME activates the corresponding expert to generate output tokens in each reasoning stage via output-oriented activation. To empower CoME with hybrid-capabilities reasoning, we introduce a progressive training strategy: Expert-FT enables decoupling and enhancement of different experts' capability; Router-FT aligns expert activation with the different reasoning stage; CoT-FT facilitates seamless collaboration and balanced optimization across multiple capabilities. To mitigate error propagation in hybrid-capabilities reasoning, we propose InfoGain-Driven DPO (Info-DPO), which uses information gain to evaluate the contribution of each intermediate step, thereby guiding CoME toward more informative reasoning. Comprehensive experiments show that CoME outperforms dense mobile agents and MoE methods on both AITZ and AMEX datasets.
Engineering Breakdown
Plain English
CoME introduces a mobile agent architecture that breaks down autonomous task execution into four specialized experts, each handling a distinct reasoning stage: screen summarization, subtask planning, action decision-making, and action execution. The paper addresses a fundamental problem in mobile agents—how to simultaneously improve individual reasoning capabilities while maintaining coherent integration across all stages. The authors propose a progressive training strategy with Expert-FT for decoupled capability enhancement and Router-FT for balanced expert coordination, enabling the system to activate the appropriate expert at each reasoning step through output-oriented activation rather than forcing all capabilities through a single model.
Core Technical Contribution
The core innovation is the Channel-of-Mobile-Experts architecture, which replaces the monolithic agent paradigm with a modular, stage-aligned expert system where each expert specializes in a single reasoning capability rather than attempting to be generalist. Unlike prior mixture-of-experts approaches that distribute computation across similar capacity experts, CoME explicitly aligns each expert to a well-defined reasoning stage in the mobile task execution pipeline, creating semantic separation between capabilities. The output-oriented activation mechanism is novel—instead of routing at the input level or using hidden representations, CoME routes by selecting which expert generates the next token sequence for each stage, creating clear boundaries between reasoning phases. The progressive training strategy with Expert-FT followed by Router-FT decouples the optimization of individual expert quality from the optimization of routing and integration, solving the known problem of catastrophic forgetting when jointly training mixed-objective systems.
How It Works
The system receives a mobile task (e.g., 'book a flight') and a current screen observation as input. In the screen summary stage, the screen-summary expert encodes the visual state and generates a textual representation of relevant UI elements. The subtask-planning expert then takes this summary and decomposes the overall task into executable subtasks with interdependencies. During action decision, the action-decision expert selects which subtask to execute next and determines what action on the UI will progress toward that subtask. Finally, the action-function expert generates the actual function call with parameters (e.g., 'click(x=142, y=305)'). The routing mechanism operates at stage boundaries: after each expert completes its output tokens, the router (learned during Router-FT) determines readiness to transition to the next stage and activates the corresponding next expert. During Expert-FT, each expert is fine-tuned independently on data annotated for its specific stage, allowing capability-focused optimization. During Router-FT, the routing policy is learned while keeping expert weights frozen, enabling the system to develop efficient activation patterns without degrading already-learned capabilities.
Production Impact
Deploying CoME would allow mobile automation teams to independently improve specific reasoning capabilities without retraining the entire agent. For example, if screen understanding is the bottleneck, you can improve only the screen-summary expert while keeping other experts fixed, reducing iteration time from full retraining cycles to single-expert fine-tuning. This modularity also simplifies debugging—failures can be attributed to specific reasoning stages, making root-cause analysis practical. The progressive training strategy eliminates a major pain point in agent development: the common experience where jointly training multiple objectives causes performance collapse in previously-learned skills. However, the architecture introduces latency overhead from sequential expert activation and potential latency variance from router decisions, making real-time mobile automation scenarios more challenging; you'd need to optimize batch sizes and deployment strategy to stay within acceptable response times. The system requires high-quality labeled data for each reasoning stage, which is more expensive to collect than end-to-end task demonstrations, raising data acquisition costs.
Limitations and When Not to Use This
CoME assumes that mobile task reasoning cleanly decomposes into four sequential stages, which may not hold for tasks requiring backtracking, parallel reasoning, or stage interleaving—cases where the agent must reconsider an earlier decision based on later observations. The paper doesn't address how the system handles out-of-distribution screens or UI layouts significantly different from training data; modular experts may fail more gracefully than monolithic models, or they may fail catastrophically if one expert breaks the contract for downstream experts. The approach requires careful data annotation at stage boundaries, which is labor-intensive and introduces annotation bias and inconsistency that could propagate through the pipeline. Sequential activation creates a critical dependency: errors in early stages (screen summary) are not recoverable by later stages, unlike end-to-end models that can sometimes compensate for intermediate mistakes. The paper doesn't discuss how to handle tasks where the four-stage decomposition doesn't apply naturally, such as dialogue-heavy interactions or tasks requiring real-time reactive control.
Research Context
CoME builds on the foundation of mixture-of-experts architectures (Shazeer et al., 2017) and mobile autonomous agents (recent work like Android in the Wild, MoE systems for LLMs) but extends them by aligning experts to semantic task structure rather than distributing similar operations. The work responds to limitations in prior mobile agents that attempt to handle all reasoning types within a single model, creating optimization conflicts and capability imbalance. It connects to curriculum learning and progressive training literature by demonstrating that staged training—first optimizing individual experts, then optimizing coordination—outperforms joint optimization. The research opens research directions in hierarchical agent architectures, stage-aware routing policies, and compositional reasoning systems, potentially influencing how future autonomous agents are designed across domains beyond mobile task execution.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::