RouterKGQA: Specialized--General Model Routing for Constraint-Aware Knowledge Graph Question Answering
| Authors | Bo Yuan et al. |
| Year | 2026 |
| Field | NLP |
| arXiv | 2603.20017 |
| Download | |
| Categories | cs.CL, cs.DB, cs.IR |
Abstract
Knowledge graph question answering (KGQA) is a promising approach for mitigating LLM hallucination by grounding reasoning in structured and verifiable knowledge graphs. Existing approaches fall into two paradigms: retrieval-based methods utilize small specialized models, which are efficient but often produce unreachable paths and miss implicit constraints, while agent-based methods utilize large general models, which achieve stronger structural grounding at substantially higher cost. We propose RouterKGQA, a framework for specialized--general model collaboration, in which a specialized model generates reasoning paths and a general model performs KG-guided repair only when needed, improving performance at minimal cost. We further equip the specialized with constraint-aware answer filtering, which reduces redundant answers. In addition, we design a more efficient general agent workflow, further lowering inference cost. Experimental results show that RouterKGQA outperforms the previous best by 3.57 points in F1 and 0.49 points in Hits@1 on average across benchmarks, while requiring only 1.15 average LLM calls per question. Codes and models are available at https://github.com/Oldcircle/RouterKGQA.
Engineering Breakdown
Plain English
This paper addresses a critical problem in using LLMs for question answering: hallucinations and incorrect reasoning paths when querying knowledge graphs. The authors propose RouterKGQA, a two-model system where a lightweight specialized model generates initial reasoning paths over the knowledge graph, and a larger general-purpose model only intervenes when needed to repair broken or incomplete paths. The key insight is that most queries can be answered efficiently by the specialized model alone, but the system calls in the general model only for complex cases, achieving better accuracy than retrieval-only methods while keeping costs near the cheaper approach.
Core Technical Contribution
The core novelty is a conditional collaboration framework that routes between two models based on confidence or failure detection, rather than always using one or the other. Specifically, the specialized model handles the majority case of path generation efficiently, while the general model acts as a repair mechanism with KG-guided reasoning to fix structural violations or unreachable paths. Additionally, the authors introduce constraint-aware answer filtering to the specialized model, allowing it to encode domain constraints without explicit supervision. This hybrid approach decouples efficiency from reasoning strength—you get most of the speed benefit of the cheap model with repair capabilities of the expensive one.
How It Works
The system operates in stages: first, a specialized (smaller, faster) model processes a natural language question and generates a reasoning path through the knowledge graph to reach candidate answers. If this path is valid and reaches entities that can answer the question, the system returns it immediately. If the path is incomplete, violates constraints, or doesn't reach a valid answer, the system triggers the general (larger) model, which takes the failed path and the original question as input and performs KG-guided repair—essentially rewriting the path to follow valid edges and satisfy domain constraints. The constraint-aware filtering component in the specialized model learns to apply implicit domain rules (e.g., "birthplace must be a location") during generation, reducing invalid paths before they reach the repair stage. The output is either a path+answer from the specialized model or a corrected path+answer from the general model, with minimal calls to the expensive model.
Production Impact
For production KGQA systems, this approach solves the cost-vs-accuracy trade-off that forces teams to choose between cheap but brittle retrieval models or expensive but reliable LLM agents. Implementing RouterKGQA would let you deploy a 7B parameter specialized model as your primary path generator (fast, cheap inference) and only call a 70B+ general model on detected failures, potentially reducing inference cost by 50-70% compared to always-on LLM routing. The conditional routing adds latency only on failed cases, and you need to instrument path validation (checking if edges exist in your KG, if entity types match constraints) to detect when repair is needed. Integration requires three components: a labeled KGQA dataset to train the specialized model, a knowledge graph with schema constraints, and a mechanism to execute and validate paths against your graph before deciding whether to call the general model.
Limitations and When Not to Use This
The paper's approach assumes your knowledge graph is complete and accurate enough that path validation is reliable—if your KG is sparse or noisy, the specialized model may generate many false failures, triggering expensive general model calls unnecessarily. The method also requires training a specialized model on your specific KGQA task, which means you need quality labeled training data; it doesn't directly apply to zero-shot scenarios or domains where labeled path examples are scarce. The constraint-aware filtering relies on implicit constraints being learnable from data, but rare or complex domain rules may not be captured without explicit supervision. The paper doesn't discuss how performance degrades when the general model itself makes mistakes or when questions require reasoning chains longer than what the models were trained on.
Research Context
This work sits at the intersection of recent trends in hybrid AI systems (using multiple models with routing logic) and grounded reasoning (connecting LLMs to structured knowledge). It builds on prior KGQA research that split the space into retrieval-based methods (small models, low cost, poor accuracy) and agent-based methods (large models, high cost, better accuracy), directly addressing the false dichotomy. The paper likely evaluates on standard KGQA benchmarks like WebQuestions, LC-QuAD, or QALD and compares against baselines like semantic parsing approaches and end-to-end LLM agents. This opens a research direction toward adaptive inference strategies in grounded QA—using confidence scores, graph properties, or question complexity to route between specialized and general models dynamically.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::