Magicremover: Tuning-free Text-guided Image inpainting with Diffusion Models
This addresses the problem of realistic image inpainting for users by overcoming data collection and training costs, though it is incremental as it builds on existing diffusion models.
The paper tackles image inpainting by proposing MagicRemover, a tuning-free method that uses diffusion models with text guidance to fill missing pixels, achieving significant improvements in quality as shown through quantitative evaluation and user studies.
Image inpainting aims to fill in the missing pixels with visually coherent and semantically plausible content. Despite the great progress brought from deep generative models, this task still suffers from i. the difficulties in large-scale realistic data collection and costly model training; and ii. the intrinsic limitations in the traditionally user-defined binary masks on objects with unclear boundaries or transparent texture. In this paper, we propose MagicRemover, a tuning-free method that leverages the powerful diffusion models for text-guided image inpainting. We introduce an attention guidance strategy to constrain the sampling process of diffusion models, enabling the erasing of instructed areas and the restoration of occluded content. We further propose a classifier optimization algorithm to facilitate the denoising stability within less sampling steps. Extensive comparisons are conducted among our MagicRemover and state-of-the-art methods including quantitative evaluation and user study, demonstrating the significant improvement of MagicRemover on high-quality image inpainting. We will release our code at https://github.com/exisas/Magicremover.