Don't Retrieve, Navigate: Distilling Enterprise Knowledge into Navigable Agent Skills for QA and RAG
| Authors | Yiqun Sun et al. |
| Year | 2026 |
| HF Upvotes | 5 |
| arXiv | 2604.14572 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
Retrieval-Augmented Generation (RAG) grounds LLM responses in external evidence but treats the model as a passive consumer of search results: it never sees how the corpus is organized or what it has not yet retrieved, limiting its ability to backtrack or combine scattered evidence. We present Corpus2Skill, which distills a document corpus into a hierarchical skill directory offline and lets an LLM agent navigate it at serve time. The compilation pipeline iteratively clusters documents, generates LLM-written summaries at each level, and materializes the result as a tree of navigable skill files. At serve time, the agent receives a bird's-eye view of the corpus, drills into topic branches via progressively finer summaries, and retrieves full documents by ID. Because the hierarchy is explicitly visible, the agent can reason about where to look, backtrack from unproductive paths, and combine evidence across branches. On WixQA, an enterprise customer-support benchmark for RAG, Corpus2Skill outperforms dense retrieval, RAPTOR, and agentic RAG baselines across all quality metrics.
Engineering Breakdown
Plain English
Corpus2Skill addresses a fundamental limitation of traditional RAG systems: LLMs retrieve documents but never understand the corpus structure itself, making it impossible to backtrack or synthesize scattered evidence. The authors propose an offline compilation pipeline that transforms a document corpus into a hierarchical skill directory—a tree structure with progressively finer summaries at each level. At inference time, the LLM agent navigates this hierarchy from high-level topics down to specific documents, giving it explicit visibility into what exists in the corpus and what it hasn't yet explored. This explicit hierarchy enables more strategic retrieval and better evidence combination than passive, query-only RAG systems.
Core Technical Contribution
The core innovation is treating corpus organization as a learnable, navigable structure rather than an opaque search index. The authors introduce a three-phase offline compilation process: iterative document clustering at multiple granularity levels, LLM-generated summaries at each cluster level, and materialization as a traversable tree of skill files indexed by topic. This contrasts sharply with standard RAG, which performs flat similarity search without showing the agent the corpus topology. The approach gives the agent agency—it can explore the hierarchy, reason about coverage gaps, and backtrack to explore different branches, mimicking how humans navigate unfamiliar knowledge domains.
How It Works
The compilation pipeline operates offline and produces a tree structure that the agent navigates at serve time. First, documents are hierarchically clustered using a bottom-up approach: fine-grained clusters at the leaf level are progressively merged into coarser topic groups, creating multiple levels of abstraction. At each clustering level, an LLM generates a concise summary that describes the cluster's contents and key themes. These summaries, along with document IDs, are materialized into skill files—structured files representing nodes in the corpus tree. At inference time, the agent receives a bird's-eye view (root-level summary), uses it to formulate a navigation strategy, then traverses the tree by requesting summaries of progressively finer subtopics. When the agent reaches a relevant leaf cluster, it retrieves the full documents by ID and incorporates them into the LLM context. The explicit hierarchy allows the agent to track what it has explored, reason about unexplored branches, and make strategic decisions about which paths to follow.
Production Impact
This approach fundamentally changes how RAG systems reason about retrieval. Instead of treating search as a black-box operation ('give me the top-k documents for this query'), engineers get a structured exploration capability where the model actively navigates a known corpus landscape. In production, this reduces hallucination and false retrieval by forcing the agent to commit to a traversal path and see corpus coverage explicitly. The trade-off is a one-time offline cost: clustering and summarization scales with corpus size, requiring significant LLM API calls or compute during the compilation phase. For a large corpus (millions of documents), building the hierarchy might require hours of processing, but the resulting skill files enable cheaper, more accurate serving—agents make fewer retrieval requests because they navigate strategically rather than relying on multiple search attempts. Latency at serve time increases slightly due to hierarchical traversal, but this is offset by fewer total LLM calls to retrieve irrelevant documents. Integration requires pre-processing pipelines to build and version the skill directory, and the system is best suited for static or slowly-evolving corpora (e.g., documentation, knowledge bases) rather than real-time data streams.
Limitations and When Not to Use This
The paper assumes the corpus is static or changes slowly enough that periodic recompilation is feasible; highly dynamic domains (news, social media) would require expensive frequent rebuilds. The hierarchical structure is created via clustering and summarization, which may miss cross-cutting topics or domain-specific organization that human curators would impose—the hierarchy is data-driven, not semantically optimized. The approach also depends on the quality of the LLM-generated summaries at each level; weak summaries lead to poor navigation and missed relevant documents. There is no analysis of how hierarchy depth affects performance, what happens when the agent gets lost in a deep tree, or how to handle ambiguous queries that span multiple branches. The paper does not compare against simpler alternatives (e.g., sparse retrieval with dense re-ranking) or measure the actual offline computation cost in wall-clock time and token usage for realistic corpus sizes. Follow-up work should explore adaptive hierarchies that adjust based on query patterns, and hybrid approaches that combine the explicit navigation with fallback dense retrieval.
Research Context
This work builds on the RAG paradigm established by Lewis et al. (2020) and addresses a known limitation: passive document retrieval without corpus awareness. It connects to research on retrieval-in-context, where models learn to reason about available evidence, and to hierarchical information retrieval (e.g., Vespa, Elasticsearch), but applies LLM-driven summarization to make hierarchies navigable by agents. The paper implicitly builds on work in hierarchical text clustering and LLM-based summarization at scale. It opens a research direction toward agent-driven, structured retrieval where the corpus topology is first-class and the retrieval process is iterative and explainable rather than a single shot. Future work likely extends this to multi-modal corpora, adaptive hierarchies that evolve based on query patterns, and integration with planning frameworks where the agent explicitly reasons about evidence gaps.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::