Learning Sparse Masks for Diffusion-based Image Inpainting
This work addresses the need for fast encoding in applications like image compression by making diffusion-based inpainting more efficient, though it is incremental as it builds on existing inpainting methods.
The paper tackles the challenge of optimizing spatial locations for known data (inpainting masks) in diffusion-based image inpainting, which is slow with stochastic methods, by proposing a learned mask generation model that achieves competitive quality with up to four orders of magnitude acceleration.
Diffusion-based inpainting is a powerful tool for the reconstruction of images from sparse data. Its quality strongly depends on the choice of known data. Optimising their spatial location -- the inpainting mask -- is challenging. A commonly used tool for this task are stochastic optimisation strategies. However, they are slow as they compute multiple inpainting results. We provide a remedy in terms of a learned mask generation model. By emulating the complete inpainting pipeline with two networks for mask generation and neural surrogate inpainting, we obtain a model for highly efficient adaptive mask generation. Experiments indicate that our model can achieve competitive quality with an acceleration by as much as four orders of magnitude. Our findings serve as a basis for making diffusion-based inpainting more attractive for applications such as image compression, where fast encoding is highly desirable.