High Accuracy Face Geometry Capture using a Smartphone Video
This enables accessible, high-quality face reconstruction for applications like virtual reality or gaming, though it is incremental as it builds on existing SLAM and detection methods.
The paper tackles the problem of creating high-fidelity 3D face models from smartphone videos in unconstrained environments, achieving significant improvements in geometric accuracy and detail capture over existing baselines.
What's the most accurate 3D model of your face you can obtain while sitting at your desk? We attempt to answer this question in our work. High fidelity face reconstructions have so far been limited to either studio settings or through expensive 3D scanners. On the other hand, unconstrained reconstruction methods are typically limited by low-capacity models. Our method reconstructs accurate face geometry of a subject using a video shot from a smartphone in an unconstrained environment. Our approach takes advantage of recent advances in visual SLAM, keypoint detection, and object detection to improve accuracy and robustness. By not being constrained to a model subspace, our reconstructed meshes capture important details while being robust to noise and being topologically consistent. Our evaluations show that our method outperforms current single and multi-view baselines by a significant margin, both in terms of geometric accuracy and in capturing person-specific details important for making realistic looking models.