SaiNet: Stereo aware inpainting behind objects with generative networks
This addresses the domain-specific problem of generating plausible stereo images for applications like VR/AR, though it appears incremental.
The paper tackles the problem of stereo-consistent image inpainting behind objects by proposing an end-to-end network with a disparity loss and realistic training masks, achieving competitive results compared to previous state-of-the-art techniques.
In this work, we present an end-to-end network for stereo-consistent image inpainting with the objective of inpainting large missing regions behind objects. The proposed model consists of an edge-guided UNet-like network using Partial Convolutions. We enforce multi-view stereo consistency by introducing a disparity loss. More importantly, we develop a training scheme where the model is learned from realistic stereo masks representing object occlusions, instead of the more common random masks. The technique is trained in a supervised way. Our evaluation shows competitive results compared to previous state-of-the-art techniques.