Joint Face Completion and Super-resolution using Multi-scale Feature Relation Learning
This addresses a real-world need for repairing degraded facial images with multiple issues, though it is incremental as it builds on existing GAN and graph convolution techniques.
The paper tackled the problem of restoring low-resolution and occluded face images simultaneously, proposing a multi-scale feature graph GAN that outperforms state-of-the-art methods in face super-resolution (up to 4x or 8x) and completion on CelebA and Helen databases.
Previous research on face restoration often focused on repairing a specific type of low-quality facial images such as low-resolution (LR) or occluded facial images. However, in the real world, both the above-mentioned forms of image degradation often coexist. Therefore, it is important to design a model that can repair LR occluded images simultaneously. This paper proposes a multi-scale feature graph generative adversarial network (MFG-GAN) to implement the face restoration of images in which both degradation modes coexist, and also to repair images with a single type of degradation. Based on the GAN, the MFG-GAN integrates the graph convolution and feature pyramid network to restore occluded low-resolution face images to non-occluded high-resolution face images. The MFG-GAN uses a set of customized losses to ensure that high-quality images are generated. In addition, we designed the network in an end-to-end format. Experimental results on the public-domain CelebA and Helen databases show that the proposed approach outperforms state-of-the-art methods in performing face super-resolution (up to 4x or 8x) and face completion simultaneously. Cross-database testing also revealed that the proposed approach has good generalizability.