FSRNet: End-to-End Learning Face Super-Resolution with Facial Priors
This addresses the problem of enhancing low-resolution face images for applications in security or forensics, but it is incremental as it builds on existing super-resolution techniques with domain-specific adaptations.
The paper tackles face super-resolution by leveraging facial priors like landmark heatmaps and parsing maps to recover high-resolution images from very low-resolution inputs without requiring alignment, resulting in FSRNet and FSRGAN models that significantly outperform state-of-the-art methods both quantitatively and qualitatively.
Face Super-Resolution (SR) is a domain-specific super-resolution problem. The specific facial prior knowledge could be leveraged for better super-resolving face images. We present a novel deep end-to-end trainable Face Super-Resolution Network (FSRNet), which makes full use of the geometry prior, i.e., facial landmark heatmaps and parsing maps, to super-resolve very low-resolution (LR) face images without well-aligned requirement. Specifically, we first construct a coarse SR network to recover a coarse high-resolution (HR) image. Then, the coarse HR image is sent to two branches: a fine SR encoder and a prior information estimation network, which extracts the image features, and estimates landmark heatmaps/parsing maps respectively. Both image features and prior information are sent to a fine SR decoder to recover the HR image. To further generate realistic faces, we propose the Face Super-Resolution Generative Adversarial Network (FSRGAN) to incorporate the adversarial loss into FSRNet. Moreover, we introduce two related tasks, face alignment and parsing, as the new evaluation metrics for face SR, which address the inconsistency of classic metrics w.r.t. visual perception. Extensive benchmark experiments show that FSRNet and FSRGAN significantly outperforms state of the arts for very LR face SR, both quantitatively and qualitatively. Code will be made available upon publication.