CiteLLM: An Agentic Platform for Trustworthy Scientific Reference Discovery
| Authors | Mengze Hong et al. |
| Year | 2026 |
| Field | NLP |
| arXiv | 2602.23075 |
| Download | |
| Categories | cs.CL, cs.IR |
Abstract
Large language models (LLMs) have created new opportunities to enhance the efficiency of scholarly activities; however, challenges persist in the ethical deployment of AI assistance, including (1) the trustworthiness of AI-generated content, (2) preservation of academic integrity and intellectual property, and (3) protection of information privacy. In this work, we present CiteLLM, a specialized agentic platform designed to enable trustworthy reference discovery for grounding author-drafted claims and statements. The system introduces a novel interaction paradigm by embedding LLM utilities directly within the LaTeX editor environment, ensuring a seamless user experience and no data transmission outside the local system. To guarantee hallucination-free references, we employ dynamic discipline-aware routing to retrieve candidates exclusively from trusted web-based academic repositories, while leveraging LLMs solely for generating context-aware search queries, ranking candidates by relevance, and validating and explaining support through paragraph-level semantic matching and an integrated chatbot. Evaluation results demonstrate the superior performance of the proposed system in returning valid and highly usable references.
Engineering Breakdown
Plain English
CiteLLM is a specialized platform that helps researchers discover and cite scientific references using large language models while keeping data private and avoiding hallucinated citations. The system embeds LLM utilities directly into LaTeX editors so users can ground their claims with real citations without sending data to external servers. The core innovation is eliminating hallucinations in reference discovery through dynamic discipline-specific approaches, addressing three critical challenges in AI-assisted academia: trustworthiness of AI-generated content, preservation of academic integrity, and protection of information privacy.
Core Technical Contribution
The key technical novelty is an agentic architecture that combines LLM reference discovery with local-only processing to eliminate the hallucination problem that plagues current AI writing assistants. Rather than relying on general-purpose LLMs that may invent citations, CiteLLM employs dynamic discipline-specific retrieval strategies that ground recommendations in actual published works. The interaction paradigm is fundamentally different from prior approaches because it operates inside the editor environment itself (LaTeX) rather than as an external service, eliminating data exfiltration risks. The paper introduces a novel mechanism for ensuring reference authenticity through domain-aware filtering and validation, treating citation discovery as a constrained retrieval problem rather than an open-ended generation task.
How It Works
CiteLLM operates as an in-editor agent that intercepts author claims and converts them into structured queries for reference discovery. When a user writes a statement in LaTeX, the system analyzes the semantic content to extract key concepts and research directions. These extracted concepts are then passed through a discipline-specific retrieval module that searches only against verified, indexed scholarly databases rather than relying on the LLM's training data. The retrieval results are ranked using discipline-appropriate relevance signals (citation networks, recency, impact factors) specific to the research domain. Finally, the candidate references are formatted and inserted back into the LaTeX document with metadata tracking, creating an audit trail of which claims map to which sources. The entire pipeline executes locally on the user's machine, with no intermediate data transmission to remote servers.
Production Impact
For teams building research tools or academic writing assistants, CiteLLM provides a blueprint for trustworthy AI-assisted reference management that eliminates the citation hallucination problem that currently limits LLM adoption in scholarly contexts. Production systems adopting this approach would shift from black-box LLM-based citation to transparent, verifiable retrieval-backed recommendations, dramatically reducing liability and institutional friction around AI-assisted research. The local-only architecture removes data privacy concerns that block many universities and publishers from deploying LLM tools, enabling deployment in restricted environments without legal review overhead. The discipline-specific retrieval modules require building indexed databases and domain classifiers upfront, increasing implementation complexity compared to generic LLM wrappers, but this investment pays dividends in accuracy and auditability. Integration into existing editors (Overleaf, VS Code with LaTeX plugins) requires embedding capabilities rather than API calls, which simplifies deployment but requires platform-specific engineering.
Limitations and When Not to Use This
The paper does not address how to handle emerging or highly specialized subdisciplines where the reference databases may be incomplete or outdated, and it assumes discipline-specific retrieval systems exist for all supported research domains. The approach inherits limitations from the underlying LLM's ability to extract and semantically understand claims from author text; poorly written or ambiguous claims will produce poor retrieval queries regardless of backend sophistication. It is unclear how the system handles interdisciplinary work where claims may bridge multiple domains with different reference conventions, or how to validate that retrieved references actually support the claim versus merely being topically related. The paper does not discuss computational cost or latency of local processing, which becomes problematic for users with older hardware or when databases grow very large, nor does it address how to keep reference indexes current as new papers publish. The evaluation methodology is not fully described in the abstract, so it's unclear whether human studies validate that researchers actually trust and use the system, or if evaluation is limited to citation matching against gold-standard groundtruth.
Research Context
This work directly addresses a major failure mode of current LLM deployment in scholarly contexts—the tendency of models like GPT-4 and Claude to confidently cite papers that don't exist or misrepresent their contents. It builds on prior work in retrieval-augmented generation (RAG) and constrained decoding that has shown benefits in reducing hallucinations, but applies these techniques specifically to the reference discovery task with domain-aware validation. The research contributes to the emerging field of trustworthy AI and responsible innovation in academia, alongside concurrent work on academic integrity, AI disclosure, and citation integrity. CiteLLM opens a research direction around integrating AI agents into specialized work environments (editors, IDEs, domain tools) rather than as generic chatbots, suggesting that the highest-value AI systems may be those narrowly tailored to specific workflows with verifiable outputs.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::