Less Detail, Better Answers: Degradation-Driven Prompting for VQA
| Authors | Haoxuan Han et al. |
| Year | 2026 |
| HF Upvotes | 13 |
| arXiv | 2604.04838 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
Recent advancements in Vision-Language Models (VLMs) have significantly pushed the boundaries of Visual Question Answering (VQA).However,high-resolution details can sometimes become noise that leads to hallucinations or reasoning errors. In this paper,we propose Degradation-Driven Prompting (DDP), a novel framework that improves VQA performance by strategically reducing image fidelity to force models to focus on essential structural information. We evaluate DDP across two distinct tasks. Physical attributes targets images prone to human misjudgment, where DDP employs a combination of 80p downsampling, structural visual aids (white background masks and orthometric lines), and In-Context Learning (ICL) to calibrate the model's focus. Perceptual phenomena addresses various machine-susceptible visual anomalies and illusions, including Visual Anomaly (VA), Color (CI), Motion(MI),Gestalt (GI), Geometric (GSI), and Visual Illusions (VI).For this task, DDP integrates a task-classification stage with specialized tools such as blur masks and contrast enhancement alongside downsampling. Our experimental results demonstrate that less is more: by intentionally degrading visual inputs and providing targeted structural prompts, DDP enables VLMs to bypass distracting textures and achieve superior reasoning accuracy on challenging visual benchmarks.
Engineering Breakdown
Plain English
This paper proposes Degradation-Driven Prompting (DDP), a framework that improves Visual Question Answering by intentionally reducing image resolution and adding structural visual aids rather than using high-fidelity inputs. The key insight is counterintuitive: high-resolution details often introduce noise that causes vision-language models to hallucinate or reason incorrectly, so the authors strategically degrade images using 80p downsampling combined with white background masks and orthometric lines to refocus the model on essential structural information. They evaluate DDP on two distinct tasks—physical attribute estimation (where humans commonly misjudge) and perceptual phenomena (where machine vision fails)—using In-Context Learning to calibrate model behavior. The approach demonstrates that sometimes less visual information, when carefully selected, produces better reasoning and fewer errors than raw high-resolution data.
Core Technical Contribution
The core novelty is the insight that image degradation can be a feature, not a bug—deliberately reducing visual fidelity forces vision-language models to focus on structural information rather than spurious details that lead to hallucinations. Unlike prior work that assumes more data and higher resolution always improve performance, DDP explicitly constructs a degradation pipeline combining downsampling, structural annotations (masks and geometric aids), and In-Context Learning to manipulate what information the model attends to. This represents a paradigm shift from resolution-maximization to strategic information bottlenecking, similar to how human experts often sketch abstractions rather than memorizing fine details. The technical contribution is a composable framework showing that image degradation + structural priors + ICL can outperform end-to-end processing on tasks where models are susceptible to detail-driven errors.
How It Works
The DDP framework takes a high-resolution input image and applies a multi-stage degradation pipeline. First, the image is downsampled to 80p resolution, which removes high-frequency noise and spurious details while preserving coarse structural information. Second, the authors augment this degraded image with structural visual aids: white background masks that isolate objects of interest and orthometric lines that emphasize geometric relationships and symmetries. These aids act as inductive biases that guide the model's attention toward task-relevant structure. Third, the degraded image plus structural aids are fed into a vision-language model via In-Context Learning prompts that provide exemplars of correct reasoning on similar tasks, calibrating the model's interpretation. The final stage is the VLM's visual question answering head, which processes this curated input representation to generate answers. The key interaction is that each component—downsampling, structural aids, and ICL—work together to create an information bottleneck that prevents the model from overweighting fine details.
Production Impact
For production VQA systems, DDP offers a direct lever to improve accuracy on downstream tasks by modifying the image preprocessing pipeline rather than retraining models. Engineers building quality control or inspection systems (e.g., manufacturing defect detection, medical imaging review) could adopt this approach by inserting a degradation stage before sending images to a VLM API, potentially reducing API costs and latency while improving accuracy. The 80p downsampling would reduce image transmission bandwidth by 60-80x compared to standard resolutions, directly lowering compute and network overhead. However, this approach requires task-specific tuning: the white background masks and orthometric lines must be generated per-domain (manufacturing tolerances look different from medical anatomy), adding a data preparation step. The trade-off is that you sacrifice generalist VLM capability in exchange for specialized accuracy on domains where detail causes hallucination, making DDP best suited for vertical applications with well-defined image types rather than open-ended web-scale VQA.
Limitations and When Not to Use This
The paper does not address how to automatically determine when degradation helps versus hurts—the framework assumes you know a priori which tasks suffer from detail-driven errors, limiting its plug-and-play applicability. Generating structural visual aids (masks, orthometric lines) requires domain expertise or manual annotation, which doesn't scale to arbitrary image types or dynamically generated content. The approach is only evaluated on two specific tasks (physical attributes and perceptual phenomena), leaving unclear how it generalizes to other VQA categories like scene understanding, counting, or reasoning over multiple objects where detail might actually be necessary. The paper also doesn't explore the interaction between degradation level and model scale—it's unclear whether larger, more capable VLMs benefit from the same 80p degradation or whether they need different thresholds. Finally, the reliance on In-Context Learning adds prompt engineering overhead and may not transfer across different VLM architectures or API providers, reducing reproducibility in production settings.
Research Context
This work builds on the recent empirical finding that vision-language models like CLIP, LLaVA, and GPT-4V suffer from object hallucinations and detail-driven errors despite their scale and capabilities. It relates to broader research on prompt engineering (chain-of-thought, in-context learning) and image preprocessing for vision tasks, as well as emerging work on the failure modes of VLMs under high-resolution inputs. The paper implicitly challenges the prevailing assumption in computer vision that resolution and fidelity are monotonically good, similar to recent findings on the limits of scaling laws and the necessity of inductive biases in neural networks. The evaluation framework on domain-specific VQA tasks (physical attributes, perceptual phenomena) opens a new direction for studying task-aware image degradation as a form of structured prompting, complementing work on visual reasoning benchmarks like VCR, GQA, and TextVQA.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::