SiENet: Siamese Expansion Network for Image Extrapolation
This addresses image outpainting for computer vision applications, offering an incremental improvement over existing methods by enhancing encoder capabilities.
The paper tackles the problem of image extrapolation (outpainting) by proposing SiENet, a two-stage siamese adversarial model that uses adaptive filling convolution and siamese adversarial mechanisms to predict unknown border content, achieving state-of-the-art results on four datasets with realistic outputs.
Different from image inpainting, image outpainting has relative less context in the image center to capture and more content at the image border to predict. Therefore, classical encoder-decoder pipeline of existing methods may not predict the outstretched unknown content perfectly. In this paper, a novel two-stage siamese adversarial model for image extrapolation, named Siamese Expansion Network (SiENet) is proposed. In two stages, a novel border sensitive convolution named adaptive filling convolution is designed for allowing encoder to predict the unknown content, alleviating the burden of decoder. Besides, to introduce prior knowledge to network and reinforce the inferring ability of encoder, siamese adversarial mechanism is designed to enable our network to model the distribution of covered long range feature for that of uncovered image feature. The results on four datasets has demonstrated that our method outperforms existing state-of-the-arts and could produce realistic results.