PFB-Diff: Progressive Feature Blending Diffusion for Text-driven Image Editing
This work addresses image editing challenges for users needing precise, artifact-free modifications in applications like photo editing, though it is incremental as it builds on existing diffusion models.
The paper tackles the problem of undesired artifacts in diffusion-based local image editing by proposing PFB-Diff, which uses progressive feature blending and an attention masking mechanism to achieve semantic coherence and high-quality edits, demonstrating superior performance in tasks like object replacement without requiring fine-tuning.
Diffusion models have demonstrated their ability to generate diverse and high-quality images, sparking considerable interest in their potential for real image editing applications. However, existing diffusion-based approaches for local image editing often suffer from undesired artifacts due to the latent-level blending of the noised target images and diffusion latent variables, which lack the necessary semantics for maintaining image consistency. To address these issues, we propose PFB-Diff, a Progressive Feature Blending method for Diffusion-based image editing. Unlike previous methods, PFB-Diff seamlessly integrates text-guided generated content into the target image through multi-level feature blending. The rich semantics encoded in deep features and the progressive blending scheme from high to low levels ensure semantic coherence and high quality in edited images. Additionally, we introduce an attention masking mechanism in the cross-attention layers to confine the impact of specific words to desired regions, further improving the performance of background editing and multi-object replacement. PFB-Diff can effectively address various editing tasks, including object/background replacement and object attribute editing. Our method demonstrates its superior performance in terms of editing accuracy and image quality without the need for fine-tuning or training. Our implementation is available at https://github.com/CMACH508/PFB-Diff.