Dual-View Training for Instruction-Following Information Retrieval
| Authors | Qingcheng Zeng et al. |
| Year | 2026 |
| HF Upvotes | 10 |
| arXiv | 2604.18845 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
Instruction-following information retrieval (IF-IR) studies retrieval systems that must not only find documents relevant to a query, but also obey explicit user constraints such as required attributes, exclusions, or output preferences. However, most retrievers are trained primarily for semantic relevance and often fail to distinguish documents that match the topic from those that satisfy the instruction. We propose a dual-view data synthesis strategy based on polarity reversal: given a query, a document that is relevant under the instruction, and a hard negative that matches the query but violates the instruction, we prompt an LLM to generate a complementary instruction under which the two documents swap relevance labels. By presenting the same document pair under complementary instructions that invert their relevance labels, the training signal forces the retriever to reconsider the same candidate set through the instruction, rather than relying on fixed topical cues. On a 305M-parameter encoder, our method improves performance on the FollowIR benchmark by 45%, surpassing general-purpose embedding models of comparable or larger scale. Through head-to-head comparisons at matched data budgets, we further show that data diversity and instruction supervision play complementary roles: the former preserves general retrieval quality, while the latter improves instruction sensitivity. These results highlight the value of targeted data synthesis for building retrieval systems that are both broadly capable and instruction-aware.
Engineering Breakdown
Plain English
This paper addresses a critical gap in information retrieval systems: most retrievers are trained only to find topically relevant documents, but fail when users have explicit constraints like required attributes or exclusions. The authors propose a dual-view training strategy that uses an LLM to generate complementary instructions for the same document pairs, creating training examples where documents swap relevance labels under different instruction contexts. This forces the retriever to learn not just semantic relevance but also instruction adherence. The core innovation is polarity reversal: if Document A matches Query + Instruction 1 but violates it, the system generates Instruction 2 where Document A becomes irrelevant and the original hard negative becomes relevant, multiplying training signal from the same data.
Core Technical Contribution
The key technical novelty is the polarity reversal data synthesis mechanism—a method to automatically generate instruction-swapped negative examples from existing triplets without human annotation. Rather than collecting new labeled data, the approach uses an LLM as a data augmentation engine to construct complementary instructions that invert document relevance relationships. This is fundamentally different from prior instruction-following retrieval work because it doesn't just mix instructions with existing data; it actively reframes the same document pairs under semantically inverted constraint contexts. The insight is that a document pair is inherently ambiguous with respect to instructions, so presenting them under multiple instruction frames forces the retriever to learn instruction-conditional relevance rather than instruction-agnostic topicality.
How It Works
The system takes three inputs: a query Q, a relevant document D+ that satisfies Instruction I, and a hard negative D- that matches the query semantically but violates I. An LLM prompt then generates a complementary instruction I' such that under I', D- becomes relevant and D+ becomes irrelevant (or at least misaligned). The retriever is then trained on both (Q, I, D+, D-) and (Q, I', D-, D+) as positive and negative pairs. This dual-view training exposes the model to the same documents from opposing instruction perspectives, forcing it to learn fine-grained conditional relevance. The training objective encourages higher scores for documents that match the current instruction context and lower scores for those that violate it, even when the documents themselves are topically similar. The LLM augmentation happens offline during data preparation, so inference latency is unaffected.
Production Impact
For production retrieval systems serving constraint-aware queries (e-commerce with filters, legal document search with exclusions, job matching with requirements), this approach directly improves ranking quality without requiring domain-specific labeled data. Teams can take existing relevance judgments and synthetic negatives, then auto-generate complementary instructions to multiply training signal, reducing annotation costs significantly. The practical benefit is that retriever models stop treating instructions as an afterthought and learn to properly distinguish 'topically relevant but instruction-violating' from 'topically relevant and instruction-compliant' documents. Trade-offs include the need for a capable LLM to generate quality instructions (OpenAI API calls or local LLM overhead), potential quality variance if generated instructions are semantically weak or redundant, and the requirement to re-train the retriever with the augmented data. Integration is relatively straightforward—add an offline data augmentation stage before existing retriever training pipelines.
Limitations and When Not to Use This
The approach assumes the LLM can reliably generate semantically distinct complementary instructions, which may fail for complex, multi-constraint scenarios or domain-specific instructions that are difficult to invert. The paper doesn't discuss how to verify that generated instructions are truly complementary or whether low-quality instruction generation degrades training; there's no analysis of instruction diversity or semantic coverage. The method also assumes hard negatives are available in the training set—it doesn't address how to construct effective negatives for new instruction types at inference time. Finally, the approach is evaluated implicitly through the abstract; without seeing full results, it's unclear whether gains hold for long-tail instructions, how much data augmentation factor is needed for diminishing returns, and whether the method scales to retrieval systems with millions of documents and hundreds of constraint types.
Research Context
This work builds on the growing body of instruction-following language model research (following the FLAN/InstructGPT direction) and applies it to the information retrieval domain, where instruction-aware ranking has emerged as an important problem for personalized and constraint-aware search. It extends prior work on hard negative mining and contrastive learning for retrieval by introducing instruction-level negatives rather than just document-level ones. The contribution fits into the broader trend of using LLMs as data generation engines to augment training sets without human annotation, alongside recent work in synthetic data generation for ML. The dual-view training concept is novel to IF-IR but draws conceptual inspiration from curriculum learning and symmetric contrastive approaches where the same example is viewed from complementary perspectives.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::