Details or Artifacts: A Locally Discriminative Learning Approach to Realistic Image Super-Resolution
This work addresses the challenge of generating realistic details without artifacts in image super-resolution, which is important for applications in computer vision and image processing, though it is incremental as it builds on existing GAN-based methods.
The paper tackles the problem of unstable training and visual artifacts in GAN-based single image super-resolution by proposing a locally discriminative learning method that discriminates between artifacts and realistic details, resulting in improved reconstruction accuracy and perceptual quality on synthetic and real-world datasets.
Single image super-resolution (SISR) with generative adversarial networks (GAN) has recently attracted increasing attention due to its potentials to generate rich details. However, the training of GAN is unstable, and it often introduces many perceptually unpleasant artifacts along with the generated details. In this paper, we demonstrate that it is possible to train a GAN-based SISR model which can stably generate perceptually realistic details while inhibiting visual artifacts. Based on the observation that the local statistics (e.g., residual variance) of artifact areas are often different from the areas of perceptually friendly details, we develop a framework to discriminate between GAN-generated artifacts and realistic details, and consequently generate an artifact map to regularize and stabilize the model training process. Our proposed locally discriminative learning (LDL) method is simple yet effective, which can be easily plugged in off-the-shelf SISR methods and boost their performance. Experiments demonstrate that LDL outperforms the state-of-the-art GAN based SISR methods, achieving not only higher reconstruction accuracy but also superior perceptual quality on both synthetic and real-world datasets. Codes and models are available at https://github.com/csjliang/LDL.