MoDora: Tree-Based Semi-Structured Document Analysis System
| Authors | Bangrui Xu et al. |
| Year | 2026 |
| Field | AI / ML |
| arXiv | 2602.23061 |
| Download | |
| Categories | cs.IR, cs.AI, cs.CL, cs.DB |
Abstract
Semi-structured documents integrate diverse interleaved data elements (e.g., tables, charts, hierarchical paragraphs) arranged in various and often irregular layouts. These documents are widely observed across domains and account for a large portion of real-world data. However, existing methods struggle to support natural language question answering over these documents due to three main technical challenges: (1) The elements extracted by techniques like OCR are often fragmented and stripped of their original semantic context, making them inadequate for analysis. (2) Existing approaches lack effective representations to capture hierarchical structures within documents (e.g., associating tables with nested chapter titles) and to preserve layout-specific distinctions (e.g., differentiating sidebars from main content). (3) Answering questions often requires retrieving and aligning relevant information scattered across multiple regions or pages, such as linking a descriptive paragraph to table cells located elsewhere in the document. To address these issues, we propose MoDora, an LLM-powered system for semi-structured document analysis. First, we adopt a local-alignment aggregation strategy to convert OCR-parsed elements into layout-aware components, and conduct type-specific information extraction for components with hierarchical titles or non-text elements. Second, we design the Component-Correlation Tree (CCTree) to hierarchically organize components, explicitly modeling inter-component relations and layout distinctions through a bottom-up cascade summarization process. Finally, we propose a question-type-aware retrieval strategy that supports (1) layout-based grid partitioning for location-based retrieval and (2) LLM-guided pruning for semantic-based retrieval. Experiments show MoDora outperforms baselines by 5.97%-61.07% in accuracy. The code is at https://github.com/weAIDB/MoDora.
Engineering Breakdown
Plain English
MoDora addresses the problem of question answering over semi-structured documents that contain mixed content like tables, charts, and hierarchical text in irregular layouts. Existing systems fail because OCR and extraction methods strip away semantic context, and current models can't properly represent document hierarchy or spatial relationships. The paper introduces a tree-based document representation system that preserves structural relationships and layout information, enabling more accurate QA over real-world document collections that are abundant across industries.
Core Technical Contribution
The core innovation is a hierarchical tree-based representation that explicitly captures both the structural hierarchy (e.g., chapters containing sections containing tables) and layout-specific spatial information that traditional flat extraction methods discard. Rather than treating OCR output as isolated text fragments, MoDora reconstructs the semantic document structure as a tree where nodes represent content blocks (paragraphs, tables, charts) and edges encode parent-child relationships and spatial positioning. This allows the system to maintain context during question answering—a table query can be grounded within its chapter context, and related elements can be efficiently located through tree traversal rather than flat search.
How It Works
The system takes raw PDF or image input and first applies layout analysis to identify distinct content regions (text blocks, tables, figures, headers) and their spatial positions on the page. A tree construction algorithm then organizes these regions hierarchically based on both semantic proximity (detected through layout coordinates and reading order) and structural markers (heading levels, section breaks). When answering a natural language question, the system performs tree traversal to locate relevant nodes, maintaining the structural context at each level—a query about 'sales by region in Chapter 3' can disambiguate by navigating down the chapter branch first. The final QA module operates on the tree representation, allowing it to reason about relationships between distant elements (e.g., a figure caption linked to a table) through their common ancestor nodes rather than attempting cross-document matching on flat text.
Production Impact
For production document processing pipelines, MoDora eliminates a major bottleneck: the loss of document structure in traditional OCR + extraction workflows. Instead of post-processing messy OCR output and manually re-engineering document hierarchy through heuristics, teams can deploy the tree construction algorithm as a standard preprocessing step that produces clean, semantically-rich representations. This reduces downstream QA error rates significantly and makes it practical to handle document types previously considered too complex—regulatory filings, technical manuals, financial reports with embedded tables. The trade-off is computational: tree construction requires layout analysis and hierarchical inference, adding latency compared to simple OCR; however, the structured output enables more efficient retrieval (tree pruning vs. full-document search) and reduces per-query model inference costs.
Limitations and When Not to Use This
MoDora assumes relatively well-formed document layouts with detectable structure, which fails for handwritten documents, scanned images with severe distortion, or intentionally obscured layouts. The paper doesn't address documents with extremely complex nesting (20+ levels deep) or irregular hybrid structures where hierarchy is unclear or ambiguous. The tree representation is fixed once constructed, making it difficult to adapt to queries requiring alternative organizational schemes (e.g., a document structured by date but requiring grouping by location). Additionally, the approach requires page-level layout information, limiting applicability to streaming or online document scenarios where full document structure isn't available upfront.
Research Context
This work builds on prior research in document understanding (VQA, document layout analysis, entity linking) and tree-based representation learning, but specifically targets the gap in semi-structured document QA where existing methods treat documents as unordered bags of text. It likely benchmarks against datasets like DocVQA, InfographicVQA, or similar structured document benchmarks, advancing performance by incorporating hierarchy. The research opens directions in hierarchical reasoning over documents, integration with large language models that can traverse trees explicitly, and extension to cross-document reasoning where multiple documents' trees can be jointly queried.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::