Training-Free Image Editing with Visual Context Integration and Concept Alignment
This addresses the need for efficient and consistent image editing for users by eliminating data collection and training costs, though it is incremental as it builds on existing pretrained models.
The paper tackles the problem of image editing requiring visual context without training costs, proposing VicoEdit, a training-free and inversion-free method that directly transforms images based on visual context and concept alignment, achieving better editing performance than state-of-the-art training-based models.
In image editing, it is essential to incorporate a context image to convey the user's precise requirements, such as subject appearance or image style. Existing training-based visual context-aware editing methods incur data collection effort and training cost. On the other hand, the training-free alternatives are typically established on diffusion inversion, which struggles with consistency and flexibility. In this work, we propose VicoEdit, a training-free and inversion-free method to inject the visual context into the pretrained text-prompted editing model. More specifically, VicoEdit directly transforms the source image into the target one based on the visual context, thereby eliminating the need for inversion that can lead to deviated trajectories. Moreover, we design a posterior sampling approach guided by concept alignment to enhance the editing consistency. Empirical results demonstrate that our training-free method achieves even better editing performance than the state-of-the-art training-based models.