Segmentation-Reconstruction-Guided Facial Image De-occlusion
This addresses the challenge of occlusions in face images for computer vision applications, offering an automated solution that is incremental over existing methods that rely on manual masks or are limited to specific occlusions.
The paper tackles the problem of removing various occlusions from face images, which degrade face-related tasks, by proposing a model that automatically removes all kinds of occlusions, including those with blurred boundaries like hairs, and demonstrates effectiveness and robustness through qualitative and quantitative results.
Occlusions are very common in face images in the wild, leading to the degraded performance of face-related tasks. Although much effort has been devoted to removing occlusions from face images, the varying shapes and textures of occlusions still challenge the robustness of current methods. As a result, current methods either rely on manual occlusion masks or only apply to specific occlusions. This paper proposes a novel face de-occlusion model based on face segmentation and 3D face reconstruction, which automatically removes all kinds of face occlusions with even blurred boundaries,e.g., hairs. The proposed model consists of a 3D face reconstruction module, a face segmentation module, and an image generation module. With the face prior and the occlusion mask predicted by the first two, respectively, the image generation module can faithfully recover the missing facial textures. To supervise the training, we further build a large occlusion dataset, with both manually labeled and synthetic occlusions. Qualitative and quantitative results demonstrate the effectiveness and robustness of the proposed method.