Learning Representations for Masked Facial Recovery
This addresses a practical issue for face recognition systems during the pandemic, but it is incremental as it builds on existing GAN inversion techniques.
The paper tackles the problem of face recognition performance decline due to masks by introducing a method for recovering unmasked face images from masked ones, showing effectiveness in unmasking and improving face verification performance on benchmark datasets.
The pandemic of these very recent years has led to a dramatic increase in people wearing protective masks in public venues. This poses obvious challenges to the pervasive use of face recognition technology that now is suffering a decline in performance. One way to address the problem is to revert to face recovery methods as a preprocessing step. Current approaches to face reconstruction and manipulation leverage the ability to model the face manifold, but tend to be generic. We introduce a method that is specific for the recovery of the face image from an image of the same individual wearing a mask. We do so by designing a specialized GAN inversion method, based on an appropriate set of losses for learning an unmasking encoder. With extensive experiments, we show that the approach is effective at unmasking face images. In addition, we also show that the identity information is preserved sufficiently well to improve face verification performance based on several face recognition benchmark datasets.