CycleNet: Rethinking Cycle Consistency in Text-Guided Diffusion for Image Manipulation
This addresses the problem of intuitive and consistent image manipulation for users of diffusion models, though it is incremental as it builds on existing methods by adding cycle consistency.
The paper tackles the challenge of enabling consistent unpaired image-to-image translation with pre-trained diffusion models by introducing CycleNet, a method that incorporates cycle consistency, and shows it generates high-quality images with superior consistency and quality, even with limited data (around 2k) and minimal resources (1 GPU).
Diffusion models (DMs) have enabled breakthroughs in image synthesis tasks but lack an intuitive interface for consistent image-to-image (I2I) translation. Various methods have been explored to address this issue, including mask-based methods, attention-based methods, and image-conditioning. However, it remains a critical challenge to enable unpaired I2I translation with pre-trained DMs while maintaining satisfying consistency. This paper introduces Cyclenet, a novel but simple method that incorporates cycle consistency into DMs to regularize image manipulation. We validate Cyclenet on unpaired I2I tasks of different granularities. Besides the scene and object level translation, we additionally contribute a multi-domain I2I translation dataset to study the physical state changes of objects. Our empirical studies show that Cyclenet is superior in translation consistency and quality, and can generate high-quality images for out-of-domain distributions with a simple change of the textual prompt. Cyclenet is a practical framework, which is robust even with very limited training data (around 2k) and requires minimal computational resources (1 GPU) to train. Project homepage: https://cyclenetweb.github.io/