A Wasserstein GAN for Joint Learning of Inpainting and Spatial Optimisation
This addresses a domain-specific problem for image restoration and compression applications, offering a novel approach but is incremental in combining GANs with spatial optimization.
The paper tackles the joint problem of image inpainting and spatial mask optimization by proposing a Wasserstein GAN that trains an inpainting generator and mask network together, resulting in significant improvements in visual quality and speed over existing methods.
Image inpainting is a restoration method that reconstructs missing image parts. However, a carefully selected mask of known pixels that yield a high quality inpainting can also act as a sparse image representation. This challenging spatial optimisation problem is essential for practical applications such as compression. So far, it has been almost exclusively adressed by model-based approaches. First attempts with neural networks seem promising, but are tailored towards specific inpainting operators or require postprocessing. To address this issue, we propose the first generative adversarial network (GAN) for spatial inpainting data optimisation. In contrast to previous approaches, it allows joint training of an inpainting generator and a corresponding mask optimisation network. With a Wasserstein distance, we ensure that our inpainting results accurately reflect the statistics of natural images. This yields significant improvements in visual quality and speed over conventional stochastic models. It also outperforms current spatial optimisation networks.