Panoramic Image Inpainting With Gated Convolution And Contextual Reconstruction Loss
This work addresses inpainting artifacts in panoramic images, which is an incremental improvement for computer vision applications like virtual reality.
The paper tackles panoramic image inpainting by proposing a framework with gated convolutions and contextual reconstruction loss to reduce artifacts, achieving higher PSNR and SSIM scores than state-of-the-art methods on the SUN360 Street View dataset.
Deep learning-based methods have demonstrated encouraging results in tackling the task of panoramic image inpainting. However, it is challenging for existing methods to distinguish valid pixels from invalid pixels and find suitable references for corrupted areas, thus leading to artifacts in the inpainted results. In response to these challenges, we propose a panoramic image inpainting framework that consists of a Face Generator, a Cube Generator, a side branch, and two discriminators. We use the Cubemap Projection (CMP) format as network input. The generator employs gated convolutions to distinguish valid pixels from invalid ones, while a side branch is designed utilizing contextual reconstruction (CR) loss to guide the generators to find the most suitable reference patch for inpainting the missing region. The proposed method is compared with state-of-the-art (SOTA) methods on SUN360 Street View dataset in terms of PSNR and SSIM. Experimental results and ablation study demonstrate that the proposed method outperforms SOTA both quantitatively and qualitatively.