StableI2I: Spotting Unintended Changes in Image-to-Image Transition
:::info Stub — Full Engineering Breakdown Coming This paper was featured on Hugging Face Daily Papers on 2026-05-06 with 10 upvotes. A full breakdown with production viability rating, implementation notes, and honest limitations is being written. Subscribe to AI Letters → :::
| Authors | Jiayang Li et al. |
| Year | 2026 |
| HF Upvotes | 10 |
| arXiv | 2605.04453 |
| Download | |
| HF Page | View on Hugging Face |
Abstract
In most real-world image-to-image (I2I) scenarios, existing evaluations primarily focus on instruction following and the perceptual quality or aesthetics of the generated images. However, they largely fail to assess whether the output image preserves the semantic correspondence and spatial structure of the input image. To address this limitation, we propose StableI2I, a unified and dynamic evaluation framework that explicitly measures content fidelity and pre--post consistency across a wide range of I2I tasks without requiring reference images, including image editing and image restoration. In addition, we construct StableI2I-Bench, a benchmark designed to systematically evaluate the accuracy of MLLMs on such fidelity and consistency assessment tasks. Extensive experimental results demonstrate that StableI2I provides accurate, fine-grained, and interpretable evaluations of content fidelity and consistency, with strong correlations to human subjective judgments. Our framework serves as a practical and reliable evaluation tool for diagnosing content consistency and benchmarking model performance in real-world I2I systems.
Engineering Breakdown
Plain English
StableI2I proposes a reference-free evaluation framework for image-to-image tasks that measures whether output images preserve the semantic content and spatial structure of inputs—a capability that existing metrics largely ignore. The authors built a benchmark to test whether multi-modal language models can accurately assess content fidelity and consistency, addressing the gap between instruction-following metrics and actual preservation of input image properties.
Key Engineering Insight
The core insight is that existing I2I evaluation metrics miss a critical failure mode: outputs that follow instructions well but silently corrupt or alter unrelated image content. By making this assessment automatic and reference-free (no ground truth needed), you can catch these drift issues at scale in production pipelines.
Why It Matters for Engineers
In production I2I systems—image editing, restoration, style transfer—users expect unrelated content to remain untouched. Current evaluation focuses on aesthetic quality or task completion, missing silent corruptions that break user trust. An automatic, referenceless checker like StableI2I catches these issues without manual review or paired datasets, critical for quality gates in real applications.
Research Context
Prior I2I evaluation relied on reference-based metrics (PSNR, SSIM, LPIPS) or just instruction-following accuracy, but couldn't detect unintended changes in non-target regions. StableI2I shifts the paradigm to dynamic, task-agnostic consistency checking using MLLMs as evaluators. This enables systematic benchmarking of fidelity across diverse I2I tasks and potentially allows model developers to optimize for stability, not just aesthetics.
:::tip Subscribe Get weekly breakdowns of papers like this in AI Letters - the newsletter for engineers building production AI systems. :::