HiMFR: A Hybrid Masked Face Recognition Through Face Inpainting
This addresses face recognition challenges in scenarios where masks occlude faces, such as during pandemics, but is incremental as it combines existing detection, inpainting, and recognition methods.
The authors tackled masked face recognition by proposing HiMFR, a hybrid system that detects masked faces, inpaints the occluded parts using a GAN, and then recognizes faces with a ViT-EfficientNetB3 model, achieving competitive performance on datasets like CelebA, SSDMNV2, MAFA, Pubfig83, and Face5.
To recognize the masked face, one of the possible solutions could be to restore the occluded part of the face first and then apply the face recognition method. Inspired by the recent image inpainting methods, we propose an end-to-end hybrid masked face recognition system, namely HiMFR, consisting of three significant parts: masked face detector, face inpainting, and face recognition. The masked face detector module applies a pretrained Vision Transformer (ViT\_b32) to detect whether faces are covered with masked or not. The inpainting module uses a fine-tune image inpainting model based on a Generative Adversarial Network (GAN) to restore faces. Finally, the hybrid face recognition module based on ViT with an EfficientNetB3 backbone recognizes the faces. We have implemented and evaluated our proposed method on four different publicly available datasets: CelebA, SSDMNV2, MAFA, {Pubfig83} with our locally collected small dataset, namely Face5. Comprehensive experimental results show the efficacy of the proposed HiMFR method with competitive performance. Code is available at https://github.com/mdhosen/HiMFR