Boosting Image Outpainting with Semantic Layout Prediction
This addresses the problem of blurry and abnormal outputs in image outpainting for computer vision applications, offering an incremental improvement over existing GAN-based methods.
The paper tackles image outpainting by decomposing it into two stages: first predicting extended semantic layouts with a GAN, then synthesizing images from these layouts, which improves handling of complex scenes. Experiments show it outperforms state-of-the-art models, generating more reasonable layouts and images.
The objective of image outpainting is to extend image current border and generate new regions based on known ones. Previous methods adopt generative adversarial networks (GANs) to synthesize realistic images. However, the lack of explicit semantic representation leads to blurry and abnormal image pixels when the outpainting areas are complex and with various objects. In this work, we decompose the outpainting task into two stages. Firstly, we train a GAN to extend regions in semantic segmentation domain instead of image domain. Secondly, another GAN model is trained to synthesize real images based on the extended semantic layouts. The first model focuses on low frequent context such as sizes, classes and other semantic cues while the second model focuses on high frequent context like color and texture. By this design, our approach can handle semantic clues more easily and hence works better in complex scenarios. We evaluate our framework on various datasets and make quantitative and qualitative analysis. Experiments demonstrate that our method generates reasonable extended semantic layouts and images, outperforming state-of-the-art models.