Uniform Attention Maps: Boosting Image Fidelity in Reconstruction and Editing
This work addresses reconstruction errors in diffusion models for image processing, offering improvements in fidelity and editing accuracy, though it appears incremental as it builds on existing tuning-free methods.
The paper tackled the problem of balancing fidelity and editing precision in tuning-free text-guided image generation and editing with diffusion models, by proposing uniform attention maps to replace cross-attention, resulting in significantly enhanced image reconstruction fidelity and robust performance in editing tasks.
Text-guided image generation and editing using diffusion models have achieved remarkable advancements. Among these, tuning-free methods have gained attention for their ability to perform edits without extensive model adjustments, offering simplicity and efficiency. However, existing tuning-free approaches often struggle with balancing fidelity and editing precision. Reconstruction errors in DDIM Inversion are partly attributed to the cross-attention mechanism in U-Net, which introduces misalignments during the inversion and reconstruction process. To address this, we analyze reconstruction from a structural perspective and propose a novel approach that replaces traditional cross-attention with uniform attention maps, significantly enhancing image reconstruction fidelity. Our method effectively minimizes distortions caused by varying text conditions during noise prediction. To complement this improvement, we introduce an adaptive mask-guided editing technique that integrates seamlessly with our reconstruction approach, ensuring consistency and accuracy in editing tasks. Experimental results demonstrate that our approach not only excels in achieving high-fidelity image reconstruction but also performs robustly in real image composition and editing scenarios. This study underscores the potential of uniform attention maps to enhance the fidelity and versatility of diffusion-based image processing methods. Code is available at https://github.com/Mowenyii/Uniform-Attention-Maps.