A Theoretical Justification for Image Inpainting using Denoising Diffusion Probabilistic Models
This work addresses the need for efficient and generalizable inpainting algorithms without retraining for each new mask, though it is incremental as it builds on existing methods.
The paper tackles the problem of image inpainting with diffusion models by analyzing RePaint, identifying a bias that hinders sample recovery, and proposes RePaint+ which provably recovers the true sample with a linear convergence rate, the first such result for diffusion-based inpainting.
We provide a theoretical justification for sample recovery using diffusion based image inpainting in a linear model setting. While most inpainting algorithms require retraining with each new mask, we prove that diffusion based inpainting generalizes well to unseen masks without retraining. We analyze a recently proposed popular diffusion based inpainting algorithm called RePaint (Lugmayr et al., 2022), and show that it has a bias due to misalignment that hampers sample recovery even in a two-state diffusion process. Motivated by our analysis, we propose a modified RePaint algorithm we call RePaint$^+$ that provably recovers the underlying true sample and enjoys a linear rate of convergence. It achieves this by rectifying the misalignment error present in drift and dispersion of the reverse process. To the best of our knowledge, this is the first linear convergence result for a diffusion based image inpainting algorithm.