Deep Structured Energy-Based Image Inpainting
This addresses the problem of filling in missing parts of images more accurately for computer vision applications, representing a strong specific gain rather than a broad paradigm shift.
The paper tackles image inpainting by proposing a structured energy-based model that learns relationships between patterns and missing regions, achieving significant improvements over state-of-the-art GAN methods, such as reducing MSE from 833.0 to 497.35 on the Olivetti face dataset and increasing PSNR from 22.3 dB to 28.4 dB on SVHN.
In this paper, we propose a structured image inpainting method employing an energy based model. In order to learn structural relationship between patterns observed in images and missing regions of the images, we employ an energy-based structured prediction method. The structural relationship is learned by minimizing an energy function which is defined by a simple convolutional neural network. The experimental results on various benchmark datasets show that our proposed method significantly outperforms the state-of-the-art methods which use Generative Adversarial Networks (GANs). We obtained 497.35 mean squared error (MSE) on the Olivetti face dataset compared to 833.0 MSE provided by the state-of-the-art method. Moreover, we obtained 28.4 dB peak signal to noise ratio (PSNR) on the SVHN dataset and 23.53 dB on the CelebA dataset, compared to 22.3 dB and 21.3 dB, provided by the state-of-the-art methods, respectively. The code is publicly available.