Generator Pyramid for High-Resolution Image Inpainting
This work provides an incremental improvement for researchers and practitioners working on high-resolution image inpainting, particularly for images with large missing regions.
This paper addresses the challenge of inpainting high-resolution images with large holes by proposing PyramidFill, a framework that disentangles content completion and texture synthesis. The model progressively completes content at lower resolutions and synthesizes textures at higher resolutions, leading to higher-quality inpainting results compared to state-of-the-art methods on datasets like CelebA-HQ, Places2, and a new NSHQ dataset.
Inpainting high-resolution images with large holes challenges existing deep learning based image inpainting methods. We present a novel framework -- PyramidFill for high-resolution image inpainting task, which explicitly disentangles content completion and texture synthesis. PyramidFill attempts to complete the content of unknown regions in a lower-resolution image, and synthesis the textures of unknown regions in a higher-resolution image, progressively. Thus, our model consists of a pyramid of fully convolutional GANs, wherein the content GAN is responsible for completing contents in the lowest-resolution masked image, and each texture GAN is responsible for synthesizing textures in a higher-resolution image. Since completing contents and synthesising textures demand different abilities from generators, we customize different architectures for the content GAN and texture GAN. Experiments on multiple datasets including CelebA-HQ, Places2 and a new natural scenery dataset (NSHQ) with different resolutions demonstrate that PyramidFill generates higher-quality inpainting results than the state-of-the-art methods. To better assess high-resolution image inpainting methods, we will release NSHQ, high-quality natural scenery images with high-resolution 1920$\times$1080.