FaceDet3D: Facial Expressions with 3D Geometric Detail Prediction
This work is significant for computer graphics and animation researchers and practitioners who need to generate realistic facial expressions with high-fidelity geometric details, addressing a limitation of existing 3DMMs.
This paper addresses the inability of Morphable Models (3DMMs) to capture fine geometric details like wrinkles and dimples caused by facial expressions. The authors introduce FaceDet3D, a method that predicts these expression-consistent 3D geometric details from a single image and uses them for photo-realistic rendering of novel expressions and views.
Facial Expressions induce a variety of high-level details on the 3D face geometry. For example, a smile causes the wrinkling of cheeks or the formation of dimples, while being angry often causes wrinkling of the forehead. Morphable Models (3DMMs) of the human face fail to capture such fine details in their PCA-based representations and consequently cannot generate such details when used to edit expressions. In this work, we introduce FaceDet3D, a first-of-its-kind method that generates - from a single image - geometric facial details that are consistent with any desired target expression. The facial details are represented as a vertex displacement map and used then by a Neural Renderer to photo-realistically render novel images of any single image in any desired expression and view. The project website is: http://shahrukhathar.github.io/2020/12/14/FaceDet3D.html