Chain-of-Thought Degrades Visual Spatial Reasoning Capabilities of Multimodal LLMs
| Authors | Sai Srinivas Kancheti et al. |
| Year | 2026 |
| HF Upvotes | 2 |
| arXiv | 2604.16060 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
Multimodal Reasoning Models (MRMs) leveraging Chain-of-Thought (CoT) based thinking have revolutionized mathematical and logical problem-solving. However, we show that this paradigm struggles with generalized spatial intelligence. We perform a comprehensive evaluation of seventeen models across thirteen spatial benchmarks and identify a critical gap: CoT prompting consistently degrades performance in visual spatial reasoning. Furthermore, through a novel No-Image++ ablation, we demonstrate that MRMs and CoT prompted MLMs suffer from severe shortcut learning, and hallucinate visual details from textual priors even when the image is absent. These findings challenge the efficacy of text-only CoT for spatial tasks and underscore the need for vision-centric reasoning paradigms.
Engineering Breakdown
Plain English
This paper evaluates seventeen multimodal reasoning models across thirteen spatial reasoning benchmarks and uncovers a critical failure mode: Chain-of-Thought prompting actually degrades performance on visual spatial tasks, despite its success in mathematical and logical reasoning. The authors introduce a novel ablation called No-Image++ that reveals models suffer from severe shortcut learning—they hallucinate visual details from text priors even when images are completely absent, suggesting they're solving spatial problems through textual reasoning rather than genuine visual understanding. The core finding is that text-centric reasoning paradigms fundamentally don't work for spatial intelligence, challenging the current dominant approach to multimodal reasoning and pointing toward the need for vision-first architectures.
Core Technical Contribution
The paper's core innovation is the No-Image++ ablation study, which systematically removes images from spatial reasoning tasks to expose shortcut learning behavior in multimodal models. Unlike prior work that assumes CoT prompting benefits all reasoning tasks equally, this work empirically demonstrates through comprehensive multi-model evaluation that this assumption breaks down specifically for spatial reasoning. The technical novelty lies in identifying and quantifying the failure mode of text-only CoT in visual domains—showing that models rely on textual statistical correlations rather than visual feature processing. This challenges the architectural assumption that unified text-centric reasoning can handle both symbolic and spatial intelligence, and opens a research direction toward vision-centric reasoning paradigms.
How It Works
The evaluation methodology involves taking pretrained multimodal reasoning models (MRMs) and testing them on spatial reasoning benchmarks in three configurations: standard image+text input, text-only input, and the novel No-Image++ condition where images are intentionally removed mid-reasoning. For each spatial benchmark, researchers prompt models with standard CoT instructions ('think step-by-step') and measure performance degradation compared to non-CoT baselines. The No-Image++ ablation works by removing visual inputs after the problem statement is encoded, forcing models to solve spatial problems using only textual descriptions of the image—if performance remains high, this reveals the model learned to exploit text priors rather than visual reasoning. By testing this across seventeen different models and thirteen spatial benchmarks, the authors establish statistical patterns showing that performance often stays constant or improves when images are removed, directly proving the shortcut learning behavior.
Production Impact
For teams building multimodal AI systems, this paper has immediate consequences for task selection and architecture choices. If your application requires genuine spatial reasoning (3D navigation, visual assembly, spatial planning), applying standard CoT prompting may actually harm performance—you need to either redesign prompting for vision-first reasoning or replace models with vision-centric architectures. Production systems currently using CoT-prompted multimodal models for spatial tasks should conduct similar No-Image++ ablations to detect whether they're getting real spatial understanding or text-based shortcut solutions. The practical cost of adopting vision-centric alternatives includes higher model complexity, potentially increased inference latency due to richer visual processing, and larger memory footprints, but the tradeoff is genuine robustness to distribution shifts in visual inputs. This finding also suggests that model evaluation pipelines for spatial applications should include ablations (removing images, corrupting images, using novel spatial configurations) to catch hidden shortcuts before deployment.
Limitations and When Not to Use This
The paper's evaluation, while comprehensive at seventeen models and thirteen benchmarks, may not generalize to proprietary closed-source models or recently released architectures designed specifically for spatial reasoning. The No-Image++ ablation reveals shortcut learning but doesn't propose a concrete alternative—it diagnoses the problem without offering a ready-made solution, leaving engineers to redesign architectures or prompting strategies. The paper assumes that high performance on text-only spatial descriptions is evidence of shortcut learning, but doesn't fully explore whether some specialized models might legitimately solve spatial problems through superior language understanding. Additionally, the work focuses on static spatial reasoning benchmarks; it's unclear how findings transfer to dynamic spatial reasoning (video understanding, real-time navigation) or to domain-specific spatial tasks (medical imaging, scientific visualization) where visual priors may be less misleading.
Research Context
This work extends the recent boom in Chain-of-Thought reasoning research (which revolutionized mathematical and logical benchmarks like GSM8K and MATH) by identifying a critical boundary condition—CoT prompting doesn't universally improve all reasoning types. It builds on prior multimodal reasoning work (like LLaVA, GPT-4V) that assumed text-based reasoning could handle visual tasks through proper prompting, challenging that assumption empirically. The paper contributes to the growing understanding of shortcut learning in deep neural networks, similar to prior work on texture bias and spurious correlations, but extends those insights into the multimodal reasoning regime. This opens a research direction toward vision-centric reasoning paradigms (potentially vision transformers with spatial-specific training objectives) as an alternative to the text-dominant approach that has dominated recent scaling efforts.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::