Diffusion Brush: A Latent Diffusion Model-based Editing Tool for AI-generated Images
This addresses the need for better editing tools for artists and users of text-to-image models, though it is incremental as it builds on existing diffusion techniques.
The paper tackles the problem of undesirable artifacts in AI-generated images by introducing Diffusion Brush, a tool that efficiently fine-tunes specific regions using latent diffusion models, with evaluation showing improved usability and effectiveness compared to existing methods.
Text-to-image generative models have made remarkable advancements in generating high-quality images. However, generated images often contain undesirable artifacts or other errors due to model limitations. Existing techniques to fine-tune generated images are time-consuming (manual editing), produce poorly-integrated results (inpainting), or result in unexpected changes across the entire image (variation selection and prompt fine-tuning). In this work, we present Diffusion Brush, a Latent Diffusion Model-based (LDM) tool to efficiently fine-tune desired regions within an AI-synthesized image. Our method introduces new random noise patterns at targeted regions during the reverse diffusion process, enabling the model to efficiently make changes to the specified regions while preserving the original context for the rest of the image. We evaluate our method's usability and effectiveness through a user study with artists, comparing our technique against other state-of-the-art image inpainting techniques and editing software for fine-tuning AI-generated imagery.