Generative Landmarks Guided Eyeglasses Removal 3D Face Reconstruction
This addresses the challenge of reconstructing 3D faces from obstructed images for applications like virtual reality or biometrics, though it appears incremental as it builds on existing 3DMM frameworks.
The paper tackles the problem of single-view 3D face reconstruction from images with eyeglasses, presenting a method that removes eyeglasses to generate photo-realistic 3D faces in-the-wild, achieving superior performance over existing methods in experiments.
Single-view 3D face reconstruction is a fundamental Computer Vision problem of extraordinary difficulty. Current systems often assume the input is unobstructed faces which makes their method not suitable for in-the-wild conditions. We present a method for performing a 3D face that removes eyeglasses from a single image. Existing facial reconstruction methods fail to remove eyeglasses automatically for generating a photo-realistic 3D face "in-the-wild".The innovation of our method lies in a process for identifying the eyeglasses area robustly and remove it intelligently. In this work, we estimate the 2D face structure of the reasonable position of the eyeglasses area, which is used for the construction of 3D texture. An excellent anti-eyeglasses face reconstruction method should ensure the authenticity of the output, including the topological structure between the eyes, nose, and mouth. We achieve this via a deep learning architecture that performs direct regression of a 3DMM representation of the 3D facial geometry from a single 2D image. We also demonstrate how the related face parsing task can be incorporated into the proposed framework and help improve reconstruction quality. We conduct extensive experiments on existing 3D face reconstruction tasks as concrete examples to demonstrate the method's superior regulation ability over existing methods often break down.