Paper Circle: An Open-source Multi-agent Research Discovery and Analysis Framework
| Authors | Komal Kumar et al. |
| Year | 2026 |
| HF Upvotes | 28 |
| arXiv | 2604.06170 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
The rapid growth of scientific literature has made it increasingly difficult for researchers to efficiently discover, evaluate, and synthesize relevant work. Recent advances in multi-agent large language models (LLMs) have demonstrated strong potential for understanding user intent and are being trained to utilize various tools. In this paper, we introduce Paper Circle, a multi-agent research discovery and analysis system designed to reduce the effort required to find, assess, organize, and understand academic literature. The system comprises two complementary pipelines: (1) a Discovery Pipeline that integrates offline and online retrieval from multiple sources, multi-criteria scoring, diversity-aware ranking, and structured outputs; and (2) an Analysis Pipeline that transforms individual papers into structured knowledge graphs with typed nodes such as concepts, methods, experiments, and figures, enabling graph-aware question answering and coverage verification. Both pipelines are implemented within a coder LLM-based multi-agent orchestration framework and produce fully reproducible, synchronized outputs including JSON, CSV, BibTeX, Markdown, and HTML at each agent step. This paper describes the system architecture, agent roles, retrieval and scoring methods, knowledge graph schema, and evaluation interfaces that together form the Paper Circle research workflow. We benchmark Paper Circle on both paper retrieval and paper review generation, reporting hit rate, MRR, and Recall at K. Results show consistent improvements with stronger agent models. We have publicly released the website at https://papercircle.vercel.app/ and the code at https://github.com/MAXNORM8650/papercircle.
Engineering Breakdown
Plain English
Paper Circle is a multi-agent LLM system that solves the problem of information overload in scientific research by automating the discovery, evaluation, and synthesis of academic papers. The system uses two complementary pipelines: a Discovery Pipeline that retrieves papers from multiple sources, scores them across multiple criteria, ranks results with diversity awareness, and outputs structured data; and an Analysis Pipeline that processes individual papers for deeper understanding. This addresses a real pain point for researchers who spend significant time manually searching, filtering, and organizing literature. The multi-agent approach leverages recent advances in LLMs' ability to understand user intent and tool-use, making the system more practical for production deployment.
Core Technical Contribution
The core innovation is a two-pipeline architecture that separates the problem of paper discovery from analysis, each optimized for its specific constraints. The Discovery Pipeline introduces multi-criteria scoring combined with diversity-aware ranking, which moves beyond simple relevance matching to surface papers that are both relevant and cover complementary perspectives—this prevents the system from returning homogeneous results. The Analysis Pipeline transforms raw papers into structured, actionable insights using multi-agent orchestration, allowing different specialized agents to handle different aspects of paper understanding (methodology, results, limitations, etc.). This separation of concerns is novel because prior work typically treated discovery and understanding as a single problem, whereas this system recognizes they have different optimization targets.
How It Works
The Discovery Pipeline begins with offline and online retrieval from multiple academic sources, creating a candidate pool. Each candidate is then scored across multiple dimensions (relevance to query, methodological quality, novelty, citation impact, etc.) using LLM-based evaluation or structured scoring functions. The multi-criteria scores feed into a diversity-aware ranking algorithm that balances relevance with coverage—so if many papers cover similar ground, the ranker deprioritizes duplicative ones in favor of papers that add new perspectives. The system outputs structured results (metadata, summaries, scores) that can be programmatically consumed. The Analysis Pipeline then takes selected papers and routes them to specialized agents: one may extract methodology details, another may identify experimental results, another may flag limitations or failure cases. These agents coordinate to produce a unified analysis document, with the coordination likely implemented via an agentic loop that reflects on partial results and iterates.
Production Impact
In production, this would dramatically reduce manual literature review time for researchers, saving hours per week on paper evaluation and synthesis. Integration would require: (1) building a paper ingestion pipeline that can handle PDFs and metadata from multiple academic sources, (2) implementing a low-latency scoring service that evaluates papers across criteria without requiring full re-reading by LLMs each time, and (3) managing API costs since multi-agent LLM systems have high per-query costs (likely 0.50 per paper depending on model and analysis depth). Latency would be a consideration—discovery from scratch might take 30–120 seconds for a comprehensive search, which is acceptable for batch research workflows but not for interactive UI. The system would need careful prompt engineering to ensure agents don't hallucinate or misrepresent paper findings, making validation against source PDFs essential before production deployment.
Limitations and When Not to Use This
The paper abstract doesn't specify how well the multi-criteria scoring generalizes across research domains—a physics paper and a machine learning paper have very different methodological standards, so the scoring function may need domain-specific tuning. The diversity-aware ranking is mentioned but not detailed; it's unclear whether this uses embedding-space diversity (which can miss semantic novelty) or relies on agent-based analysis (which adds latency and cost). The system assumes high-quality offline sources and that online retrieval (presumably via APIs like arXiv, Scholar) covers the researcher's target domain, which breaks down for proprietary or niche research. The Analysis Pipeline's multi-agent coordination is not fully specified in the abstract—without clear failure handling, if one agent produces incorrect output, it could propagate downstream. Finally, there's no mention of how the system validates its outputs against ground truth or handles papers with novel/unconventional methodologies that might confuse standard scoring functions.
Research Context
This work builds on the recent success of multi-agent LLM systems (following papers like AutoGen and similar work from 2023–2025) and extends them to a concrete, high-impact use case. It also draws on prior work in academic paper retrieval (like Semantic Scholar, which uses citation graphs and embeddings) but advances the state by adding explicit multi-criteria evaluation and diversity ranking rather than pure relevance matching. The paper likely benchmarks against baselines like BM25 keyword search, single-agent LLM summarization, and citation-based ranking to show improvements. This opens up follow-up directions: (1) personalized scoring functions that adapt to individual researcher preferences, (2) cross-paper relationship discovery (finding how papers build on or contradict each other), and (3) meta-analysis generation that synthesizes findings across papers with uncertainty quantification.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::