Panoptic Pairwise Distortion Graph
| Authors | Muhammad Kamran Janjua et al. |
| Year | 2026 |
| HF Upvotes | 2 |
| arXiv | 2604.11004 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
In this work, we introduce a new perspective on comparative image assessment by representing an image pair as a structured composition of its regions. In contrast, existing methods focus on whole image analysis, while implicitly relying on region-level understanding. We extend the intra-image notion of a scene graph to inter-image, and propose a novel task of Distortion Graph (DG). DG treats paired images as a structured topology grounded in regions, and represents dense degradation information such as distortion type, severity, comparison and quality score in a compact interpretable graph structure. To realize the task of learning a distortion graph, we contribute (i) a region-level dataset, PandaSet, (ii) a benchmark suite, PandaBench, with varying region-level difficulty, and (iii) an efficient architecture, Panda, to generate distortion graphs. We demonstrate that PandaBench poses a significant challenge for state-of-the-art multimodal large language models (MLLMs) as they fail to understand region-level degradations even when fed with explicit region cues. We show that training on PandaSet or prompting with DG elicits region-wise distortion understanding, opening a new direction for fine-grained, structured pairwise image assessment.
Engineering Breakdown
Plain English
This paper introduces Distortion Graph (DG), a new approach to comparing image pairs by representing them as structured graphs of regions rather than analyzing whole images end-to-end. Instead of treating paired images holistically, the authors decompose them into region-level components and model the relationships between corresponding regions to capture distortion type, severity, and quality differences. The paper contributes three concrete artifacts: PandaSet (a region-level annotated dataset), PandaBench (a benchmark suite with varying difficulty levels), and a learning framework that operates on this structured representation. The key innovation is extending scene graphs—which traditionally model relationships within a single image—to work across image pairs, enabling dense, interpretable degradation information in a compact graph structure.
Core Technical Contribution
The core novelty is the Distortion Graph task itself: a structured, inter-image representation that treats paired images as a topology grounded in regions rather than flat global features. This extends the intra-image scene graph concept to pairwise comparison, where nodes represent image regions and edges encode distortion relationships, severity, and quality metrics. The authors move away from implicit region-level reasoning (as in existing methods) to explicit, structured region-level analysis with interpretable graph outputs. This is fundamentally different from prior comparative assessment methods that operate on whole images; DG makes region relationships and degradation patterns first-class citizens of the model representation.
How It Works
The approach works in stages: first, an image pair is segmented into corresponding regions (likely via panoptic segmentation, given the 'Panoptic' in the title). Each region becomes a node in a bipartite or relational graph structure. Edges between regions from image A and image B encode attributes like distortion type (blur, noise, compression, etc.), severity score, and relative quality ranking. The model learns to predict these edge attributes and node-level scores by processing visual features from paired regions through an encoder that produces graph-structured outputs. The resulting Distortion Graph is compact (fewer parameters than pixel-level comparisons) and interpretable (you can trace which regions have which degradations and by how much). The structured output allows both dense quality prediction and symbolic reasoning about where and why images differ.
Production Impact
For engineers building image quality assessment or comparative analysis pipelines, this enables several concrete improvements: (1) explainability—you get region-level degradation reports instead of black-box quality scores, useful for diagnostics and debugging; (2) fine-grained quality metrics tied to semantic regions, valuable for applications like photo enhancement or defect detection; (3) compact representations that could lower inference latency vs. pixel-level models; (4) transfer learning potential across datasets via region-level semantics. The main trade-off is complexity: you need region segmentation preprocessing, graph neural network inference, and annotation overhead for training data. For applications requiring interpretability (medical imaging QA, satellite image assessment) this is worth it; for simple end-to-end quality scoring on massive scale, the overhead may not justify the benefit.
Limitations and When Not to Use This
The paper's scope is limited by its reliance on accurate region segmentation—if panoptic segmentation fails, the entire graph representation breaks down, creating a cascading error problem. The approach assumes region-level correspondence is meaningful, which may not hold for images with significant structural changes or novel object appearances. The paper doesn't clearly address computational cost of graph construction and inference at scale, nor does it specify latency/throughput numbers for production deployment. Additionally, the generalization of PandaSet and PandaBench to diverse domains (medical imaging, video, 3D) is unclear; the dataset may overfit to natural image photography. The paper also doesn't discuss how to handle variable numbers of regions per image pair or how robustness degrades with partial occlusion or misalignment.
Research Context
This work builds on the scene graph literature (which has focused on intra-image relationships) and extends it to the comparative image assessment domain, which has traditionally used global metrics like SSIM, LPIPS, and learned end-to-end models. It sits between the interpretability push in vision (towards explainable AI) and the structured prediction literature (graph-based reasoning). The paper likely improves over prior image quality assessment benchmarks (like BIDS, KADIS) by providing region-level annotations and structured outputs. It opens a research direction toward explainable comparative assessment and could influence how the community thinks about quality metrics—moving from scalar outputs to rich structured representations. This positions DG as a bridge between symbolic AI (graphs, reasoning) and learned vision models.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::