ThinkOmni: Lifting Textual Reasoning to Omni-modal Scenarios via Guidance Decoding
| Authors | Yiran Guan et al. |
| Year | 2026 |
| Field | Computer Vision |
| arXiv | 2602.23306 |
| Download | |
| Categories | cs.CV |
Abstract
Omni-modal reasoning is essential for intelligent systems to understand and draw inferences from diverse data sources. While existing omni-modal large language models (OLLM) excel at perceiving diverse modalities, they lack the complex reasoning abilities of recent large reasoning models (LRM). However, enhancing the reasoning ability of OLLMs through additional training presents significant challenges, including the need for high-quality data, task-specific adaptation, and substantial computational costs. To address these limitations, we propose ThinkOmni, a training-free and data-free framework that lifts textual reasoning to omni-modal scenarios. ThinkOmni introduces two key components: 1) LRM-as-a-Guide, which leverages off-the-shelf LRMs to guide the OLLM decoding process; 2) Stepwise Contrastive Scaling, which adaptively balances perception and reasoning signals without manual hyperparameter tuning. Experiments on six multi-modal reasoning benchmarks demonstrate that ThinkOmni consistently delivers performance improvements, with main results achieving 70.2 on MathVista and 75.5 on MMAU. Overall, ThinkOmni offers a flexible and generalizable solution for omni-modal reasoning and provides new insights into the generalization and application of reasoning capabilities.
Engineering Breakdown
Plain English
ThinkOmni addresses a critical gap in omni-modal large language models (OLLMs): while they can perceive multiple data types (images, audio, video, text), they lack the sophisticated reasoning abilities of recent large reasoning models (LRMs). The paper proposes a training-free, data-free framework that leverages off-the-shelf LRMs to guide OLLM decoding without requiring expensive retraining, new datasets, or additional computational overhead during model development. The core innovation is using an LRM-as-a-Guide component that steers the OLLM's token generation process, effectively transferring reasoning capability across modalities without modifying either model. This approach makes advanced reasoning accessible to resource-constrained practitioners who cannot afford the computational and data costs of fine-tuning large models.
Core Technical Contribution
The key novelty is the decoupling of reasoning enhancement from model training through guidance decoding—using an external reasoning model to influence token selection in a frozen OLLM without any parameter updates. ThinkOmni introduces a two-stage inference mechanism where an LRM acts as a reasoning oracle that constrains or weights the OLLM's next-token probabilities based on intermediate reasoning steps. This is fundamentally different from standard approaches like fine-tuning or reinforcement learning that require backpropagation and gradient computation. The framework operates entirely at inference time, treating the OLLM and LRM as black boxes, which makes it compatible with any existing model pair and represents a shift toward post-hoc reasoning enhancement rather than pre-training or supervised adaptation.
How It Works
The system takes multimodal input (image, text, audio, etc.) and feeds it into the OLLM, which begins generating a response token-by-token. At each decoding step, the framework extracts the current token probabilities and intermediate reasoning state from the OLLM. These are passed to the LRM-as-a-Guide component, which performs internal reasoning about what the next token should be, generating a reasoning distribution over likely continuations. The OLLM's probability distribution is then reweighted or constrained using the LRM's guidance signal—effectively raising probabilities for tokens the LRM deems logical and lowering probabilities for inconsistent choices. The modified distribution is used for the next token selection, and the process repeats. This decoding-time steering preserves the OLLM's multimodal perception while layering in the LRM's reasoning capability, all without updating any model weights or requiring paired training data.
Production Impact
For teams deploying OLLMs in reasoning-heavy applications (document analysis, technical support, code generation with multimodal context), ThinkOmni offers an immediate path to higher-quality outputs without retraining infrastructure or data pipelines. You can take an existing OLLM deployment and pair it with a reasoning model at inference time—no changes to the training or data ETL layers. The trade-off is clear: inference latency increases because you must run both models, and you need enough compute to run two models in parallel or sequence, which roughly doubles memory and compute requirements per request. Integration is straightforward—this is a wrapper around the decoding loop—but you lose the ability to batch aggressively since guidance generation per token is a sequential dependency. Cost scales with inference volume: if you're already running an OLLM at scale, adding an LRM inference pass is significant, but it avoids the much higher cost of collecting multimodal reasoning datasets, annotating them, and retraining for 100B+ tokens.
Limitations and When Not to Use This
The approach assumes both the OLLM and LRM are available and responsive enough for interactive inference—if either model is slow, latency becomes prohibitive for real-time applications. It does not fundamentally improve the OLLM's multimodal perception; it only enhances reasoning on top of what the model already understands, so if the OLLM misses visual details or audio nuances, no amount of guidance will recover that information. The quality of guidance is bounded by the LRM's reasoning capability and its ability to reason about multimodal context without directly seeing images or hearing audio—the LRM receives only text representations of non-text modalities, introducing a bottleneck. The paper does not address how to handle cases where the OLLM and LRM disagree on coherence or logical consistency, nor does it provide principled methods for weighting the guidance signal, leaving hyperparameter tuning as an open question. Finally, this approach is orthogonal to but not a replacement for improving OLLMs directly; it is a stopgap that works well when reasoning is the bottleneck but does not solve perception-level failures or hallucination in multimodal understanding.
Research Context
ThinkOmni builds on two converging trends: the explosion of omni-modal models like GPT-4V and LLaVA that unify perception and language, and the recent surge of reasoning-specialized models like OpenAI o1 and DeepSeek-R1 that prioritize step-by-step reasoning over fast inference. It addresses a known limitation in the OLLM literature—that scaling model size alone does not guarantee reasoning capability, especially on novel multimodal tasks. The work is positioned as a practical alternative to the standard pipeline of training larger models with more diverse data, echoing ideas from prompting-based steering methods (like chain-of-thought guidance) but applying them to cross-model inference. This opens a research direction on composition of specialized models at inference time rather than monolithic end-to-end models, with implications for modular AI systems where perception, reasoning, and planning are decoupled components that can be swapped and combined without retraining.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::