PIINET: A 360-degree Panoramic Image Inpainting Network Using a Cube Map
This work addresses a domain-specific problem for computer vision applications involving panoramic images, representing an incremental advancement by adapting existing GAN techniques to a new format.
The authors tackled the problem of inpainting 360-degree panoramic images, which had not been actively studied with deep learning, by proposing a GAN-based method that converts images to a cube map format to reduce distortion and uses discriminative networks to handle correlations across cube faces, resulting in qualitative performance improvements over existing single-image inpainting and baseline algorithms.
Inpainting has been continuously studied in the field of computer vision. As artificial intelligence technology developed, deep learning technology was introduced in inpainting research, helping to improve performance. Currently, the input target of an inpainting algorithm using deep learning has been studied from a single image to a video. However, deep learning-based inpainting technology for panoramic images has not been actively studied. We propose a 360-degree panoramic image inpainting method using generative adversarial networks (GANs). The proposed network inputs a 360-degree equirectangular format panoramic image converts it into a cube map format, which has relatively little distortion and uses it as a training network. Since the cube map format is used, the correlation of the six sides of the cube map should be considered. Therefore, all faces of the cube map are used as input for the whole discriminative network, and each face of the cube map is used as input for the slice discriminative network to determine the authenticity of the generated image. The proposed network performed qualitatively better than existing single-image inpainting algorithms and baseline algorithms.