The Consistency Critic: Correcting Inconsistencies in Generated Images via Reference-Guided Attentive Alignment
This addresses the inconsistency issue in image generation for applications requiring high detail accuracy, but it is incremental as it builds on existing reference-guided methods.
The paper tackles the problem of inconsistent fine-grained details in generated images by proposing a reference-guided post-editing approach called ImageCritic, which uses an attention alignment loss and detail encoder to correct inconsistencies, showing significant improvements over existing methods in experiments.
Previous works have explored various customized generation tasks given a reference image, but they still face limitations in generating consistent fine-grained details. In this paper, our aim is to solve the inconsistency problem of generated images by applying a reference-guided post-editing approach and present our ImageCritic. We first construct a dataset of reference-degraded-target triplets obtained via VLM-based selection and explicit degradation, which effectively simulates the common inaccuracies or inconsistencies observed in existing generation models. Furthermore, building on a thorough examination of the model's attention mechanisms and intrinsic representations, we accordingly devise an attention alignment loss and a detail encoder to precisely rectify inconsistencies. ImageCritic can be integrated into an agent framework to automatically detect inconsistencies and correct them with multi-round and local editing in complex scenarios. Extensive experiments demonstrate that ImageCritic can effectively resolve detail-related issues in various customized generation scenarios, providing significant improvements over existing methods.