StableI2I: Spotting Unintended Changes in Image-to-Image Transition
For researchers and practitioners in image-to-image generation, it provides a practical tool to diagnose unintended content changes, addressing a gap in existing evaluations.
The paper proposes StableI2I, a unified evaluation framework that measures content fidelity and pre-post consistency in image-to-image tasks without reference images, and introduces StableI2I-Bench for benchmarking MLLMs on these tasks. Experiments show strong correlation with human judgments.
In most real-world image-to-image (I2I) scenarios, existing evaluations primarily focus on instruction following and the perceptual quality or aesthetics of the generated images. However, they largely fail to assess whether the output image preserves the semantic correspondence and spatial structure of the input image. To address this limitation, we propose StableI2I, a unified and dynamic evaluation framework that explicitly measures content fidelity and pre--post consistency across a wide range of I2I tasks without requiring reference images, including image editing and image restoration. In addition, we construct StableI2I-Bench, a benchmark designed to systematically evaluate the accuracy of MLLMs on such fidelity and consistency assessment tasks. Extensive experimental results demonstrate that StableI2I provides accurate, fine-grained, and interpretable evaluations of content fidelity and consistency, with strong correlations to human subjective judgments. Our framework serves as a practical and reliable evaluation tool for diagnosing content consistency and benchmarking model performance in real-world I2I systems.