MagicEraser: Erasing Any Objects via Semantics-Aware Control
This work addresses the object erasure task for image editing applications, representing an incremental improvement over prior diffusion-based methods.
The paper tackles the problem of object erasure in images, where existing methods often produce incongruent results or require high-quality text prompts, by introducing MagicEraser, a diffusion model-based framework that achieves fine control and reduces artifacts, as demonstrated in experimental results.
The traditional image inpainting task aims to restore corrupted regions by referencing surrounding background and foreground. However, the object erasure task, which is in increasing demand, aims to erase objects and generate harmonious background. Previous GAN-based inpainting methods struggle with intricate texture generation. Emerging diffusion model-based algorithms, such as Stable Diffusion Inpainting, exhibit the capability to generate novel content, but they often produce incongruent results at the locations of the erased objects and require high-quality text prompt inputs. To address these challenges, we introduce MagicEraser, a diffusion model-based framework tailored for the object erasure task. It consists of two phases: content initialization and controllable generation. In the latter phase, we develop two plug-and-play modules called prompt tuning and semantics-aware attention refocus. Additionally, we propose a data construction strategy that generates training data specially suitable for this task. MagicEraser achieves fine and effective control of content generation while mitigating undesired artifacts. Experimental results highlight a valuable advancement of our approach in the object erasure task.