Beyond Augmented-Action Surrogates for Multi-Expert Learning-to-Defer
| Authors | Yannis Montreuil et al. |
| Year | 2026 |
| Field | Statistics / ML |
| arXiv | 2604.09414 |
| Download | |
| Categories | stat.ML, cs.LG |
Abstract
Learning-to-Defer routes each input to the expert that minimizes expected cost, but it assumes that the information available to every expert is fixed at decision time. Many modern systems violate this assumption: after selecting an expert, one may also choose what additional information that expert should receive, such as retrieved documents, tool outputs, or escalation context. We study this problem and call it Learning-to-Defer with advice. We show that a broad family of natural separated surrogates, which learn routing and advice with distinct heads, is inconsistent even in the smallest non-trivial setting. We then introduce an augmented surrogate that operates on the composite expert--advice action space and prove an \mathcal{H}-consistency guarantee together with an excess-risk transfer bound, yielding recovery of the Bayes-optimal policy in the limit. Experiments on tabular, language, and multi-modal tasks show that the resulting method improves over standard Learning-to-Defer while adapting its advice-acquisition behavior to the cost regime; a synthetic benchmark confirms the failure mode predicted for separated surrogates.
Engineering Breakdown
Plain English
This paper tackles a limitation in Learning-to-Defer systems where the assumption that each expert has fixed information at decision time breaks down in modern applications. The authors study Learning-to-Defer with advice—where you can dynamically select not just which expert handles a task, but also what additional information (retrieved documents, tool outputs, context) that expert receives. They discover that natural approaches using separate neural network heads for routing and advice selection are mathematically inconsistent and don't guarantee good solutions, even in simple settings. They propose an augmented surrogate that operates on the combined expert-advice action space instead, proving it satisfies H-consistency guarantees with bounded excess risk.
Core Technical Contribution
The core novelty is identifying and formalizing a gap between theory and practice in Learning-to-Defer: separated surrogates (handling routing and advice as independent decisions) provably fail to converge to optimal solutions due to inconsistency. The authors introduce an augmented surrogate framework that treats the expert-advice pair as a single composite action, then prove this approach satisfies H-consistency with explicit excess risk bounds. This is a shift from treating routing and advice selection as decoupled problems to a unified decision framework. The theoretical guarantee—H-consistency with excess risk bounds—provides practitioners confidence that the learned policy approximates the optimal defer-and-advice strategy.
How It Works
The system operates in a sequential decision framework where each input must be routed to one of multiple experts, and simultaneously, contextual advice (retrieved documents, tool outputs, escalation data) must be selected for that expert. Traditional Learning-to-Defer learns a routing policy p(expert | input) independently from advice selection, using separated heads in the neural network. The augmented surrogate reframes this as learning over the joint action space A = {(expert_i, advice_j) | i ∈ experts, j ∈ advice_options}, where each composite action carries its own cost function. During training, the surrogate loss encourages the model to assign high probability to composite actions with low actual cost (measured as expert_i's performance when given advice_j), and the framework uses importance weighting or other techniques to handle the mismatch between the surrogate loss and the true Bayes-optimal composite action selection.
Production Impact
For engineers building AI systems with multiple experts (e.g., routing to different LLM sizes, specialized models, or human escalation), this enables dynamic context enrichment at decision time rather than static information pre-allocation. If you're building a document retrieval system that routes queries to different expert models and can also choose which retrieved documents each expert sees, this approach would let you learn joint routing-and-retrieval policies that minimize end-to-end cost (latency × quality). The H-consistency guarantee means you can trust convergence behavior—the learned policy won't be arbitrarily far from optimal even when using standard SGD. Trade-offs include increased action space size (if you have 10 experts and 50 advice options, your composite space is 500 actions), requiring more training data and careful hyperparameter tuning compared to separated approaches, and potential latency if advice selection is expensive (e.g., real-time document retrieval).
Limitations and When Not to Use This
The paper assumes that expert performance costs can be observed or estimated for each (expert, advice) pair, which is expensive in systems with many experts and advice options—gathering ground truth for all combinations may be infeasible. The approach doesn't address scenarios where advice options are continuous (e.g., how much context to retrieve) or where advice generation itself has latent variables, restricting applicability to discrete advice spaces. The H-consistency guarantee bounds excess risk but doesn't characterize the constant factors or provide guidance on sample complexity, so practitioners won't know a priori how much data is needed for a given problem. Follow-up work should address computational scalability (composite action space grows multiplicatively), adaptive advice generation (rather than predefined options), and empirical comparison against separated surrogates on realistic systems.
Research Context
This work extends the Learning-to-Defer framework (which routes each input to the expert minimizing expected loss) to handle dynamic information allocation—a problem motivated by modern systems like retrieval-augmented generation and multi-tool AI agents. It builds on prior consistency theory for surrogate losses (H-consistency, φ-consistency) but applies it to a new structured prediction setting with composite actions. The paper contributes to the broader research direction of moving away from treating routing and information provision as decoupled—similar to recent work on joint optimization in retrieval and ranking systems. It opens questions about how to efficiently handle exponential action spaces in defer-and-advise systems and how to extend guarantees to continuous advice selection.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::