Hybrid Approach for 3D Head Reconstruction: Using Neural Networks and Visual Geometry
This work addresses the challenge of 3D face reconstruction for computer vision applications, presenting an incremental improvement by integrating neural networks with existing geometric methods.
The paper tackles the problem of reconstructing 3D heads from single or multiple images by combining deep learning with geometric techniques, achieving state-of-the-art results in both single and multi-view settings and successfully recovering 3D geometry and precise poses for real-world images despite training only on synthetic data.
Recovering the 3D geometric structure of a face from a single input image is a challenging active research area in computer vision. In this paper, we present a novel method for reconstructing 3D heads from a single or multiple image(s) using a hybrid approach based on deep learning and geometric techniques. We propose an encoder-decoder network based on the U-net architecture and trained on synthetic data only. It predicts both pixel-wise normal vectors and landmarks maps from a single input photo. Landmarks are used for the pose computation and the initialization of the optimization problem, which, in turn, reconstructs the 3D head geometry by using a parametric morphable model and normal vector fields. State-of-the-art results are achieved through qualitative and quantitative evaluation tests on both single and multi-view settings. Despite the fact that the model was trained only on synthetic data, it successfully recovers 3D geometry and precise poses for real-world images.