Pix2face: Direct 3D Face Model Estimation
This addresses the problem of efficient 3D face modeling for applications like computer vision, but it appears incremental as it builds on existing U-Net and 3DMM frameworks.
The paper tackles 3D face shape and pose estimation from unconstrained 2D images by proposing a method that jointly estimates dense 3D landmarks and geometry using a modified U-Net, then directly computes 3D Morphable Model parameters via a linear system, with results evaluated quantitatively on video sequences.
An efficient, fully automatic method for 3D face shape and pose estimation in unconstrained 2D imagery is presented. The proposed method jointly estimates a dense set of 3D landmarks and facial geometry using a single pass of a modified version of the popular "U-Net" neural network architecture. Additionally, we propose a method for directly estimating a set of 3D Morphable Model (3DMM) parameters, using the estimated 3D landmarks and geometry as constraints in a simple linear system. Qualitative modeling results are presented, as well as quantitative evaluation of predicted 3D face landmarks in unconstrained video sequences.