EXAONE 4.5 Technical Report
| Authors | Eunbi Choi et al. |
| Year | 2026 |
| HF Upvotes | 32 |
| arXiv | 2604.08644 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
This technical report introduces EXAONE 4.5, the first open-weight vision language model released by LG AI Research. EXAONE 4.5 is architected by integrating a dedicated visual encoder into the existing EXAONE 4.0 framework, enabling native multimodal pretraining over both visual and textual modalities. The model is trained on large-scale data with careful curation, particularly emphasizing document-centric corpora that align with LG's strategic application domains. This targeted data design enables substantial performance gains in document understanding and related tasks, while also delivering broad improvements across general language capabilities. EXAONE 4.5 extends context length up to 256K tokens, facilitating long-context reasoning and enterprise-scale use cases. Comparative evaluations demonstrate that EXAONE 4.5 achieves competitive performance in general benchmarks while outperforming state-of-the-art models of similar scale in document understanding and Korean contextual reasoning. As part of LG's ongoing effort toward practical industrial deployment, EXAONE 4.5 is designed to be continuously extended with additional domains and application scenarios to advance AI for a better life.
Engineering Breakdown
Plain English
EXAONE 4.5 is LG's first open-weight vision-language model that combines a visual encoder with their existing EXAONE 4.0 text model to enable native multimodal understanding. The model was trained on carefully curated large-scale data with emphasis on document-centric corpora, allowing it to excel at document understanding tasks while maintaining strong general language performance. A key capability is its 256K token context window, which enables long-form reasoning and enterprise applications. The paper demonstrates that this targeted data curation strategy delivers both substantial gains in document-related tasks and broad improvements across general language benchmarks.
Core Technical Contribution
The core novelty is the architectural integration of a purpose-built visual encoder into an existing foundation model framework to enable end-to-end multimodal pretraining, rather than treating vision and language as separate modules. The authors introduce a data curation strategy that strategically emphasizes document-centric content—aligning with real enterprise use cases—while maintaining broad language capabilities through mixed training data. They extend the context window to 256K tokens, which required architectural modifications beyond standard transformer scaling. This combination of architectural integration, strategic data curation, and extended context represents a departure from typical vision-language model development that often uses generic multimodal datasets without domain targeting.
How It Works
The architecture starts with a visual encoder that processes images into dense feature representations, which are then projected into the same embedding space as the text model. These visual embeddings are interleaved with token embeddings from the text stream, allowing the unified transformer backbone (inherited from EXAONE 4.0) to jointly attend to both modalities during pretraining and inference. The training pipeline ingests documents containing both text and images, with careful data curation to oversample document-heavy content (likely receipts, forms, technical diagrams, and PDFs) while maintaining sufficient general web text to preserve language model capabilities. For the extended 256K context, the authors likely employed efficient attention mechanisms (such as sparse attention or rope positional embeddings) to avoid quadratic memory scaling. During inference, the model accepts image+text prompts and generates text responses, with the unified attention mechanism enabling fine-grained cross-modal reasoning within a single forward pass.
Production Impact
Teams building enterprise document processing systems would see immediate value: OCR + understanding pipelines can be replaced with a single model call for document interpretation, form extraction, and visual question answering over documents. The 256K context window is operationally significant—it means processing entire reports, contracts, or multi-page documents in a single inference call without chunking, reducing latency and enabling more coherent long-document analysis. However, the production cost is substantial: inference on documents with full 256K context will be significantly slower and more memory-intensive than smaller models, requiring GPU clusters for real-time performance. Integration requires careful handling of image preprocessing (resolution, format) and batching strategy, since mixing variable-length image and text inputs adds complexity. For teams with document-heavy workloads (financial services, legal tech, technical support), this addresses a real gap; for general chatbot applications, the investment in document capability may be wasted compute.
Limitations and When Not to Use This
The paper does not detail the computational requirements for training or inference at 256K context—a critical production consideration. It emphasizes document understanding but provides no clarity on failure modes with low-quality images, unusual document layouts, or adversarial visual inputs that might fool the encoder. The approach assumes that mixing document-centric data with general web text will preserve language capabilities, but there's no analysis of potential capability degradation or negative transfer from domain-specific curation. The open-weight release is valuable but the paper lacks discussion of safety mechanisms around the multimodal understanding—for example, can the model be misused to extract sensitive information from documents, and what safeguards are in place? Finally, the paper doesn't address how this architecture scales to longer sequences (e.g., video) or whether the design choices optimize for a specific document distribution that may not generalize to other visual domains.
Research Context
This work builds on the momentum of vision-language models like CLIP, LLaVA, and Qwen-VL, but differentiates by starting from a strong closed-source foundation (EXAONE 4.0) and strategically targeting enterprise document understanding rather than generic visual reasoning. The extended 256K context window aligns with broader industry trends toward long-context transformers (Claude 200K, GPT-4 Turbo 128K), but applies them to multimodal settings which is less explored. The strategic data curation approach is inspired by findings in LLM scaling that show quality and domain relevance matter as much as quantity, challenging the assumption that bigger generic datasets always win. This opens a research direction on how to effectively combine vision and language in domain-targeted models without losing generalization—a practically important problem as enterprises demand specialized capabilities.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::