Detailed Facial Geometry Recovery from Multi-View Images by Learning an Implicit Function
This work addresses the need for efficient and accurate 3D face reconstruction for applications in computer vision and graphics, representing an incremental improvement over existing learning-based methods.
The paper tackles the problem of recovering detailed 3D facial geometry from multi-view images by proposing a novel architecture that learns an implicit function for regressing matching cost, achieving state-of-the-art accuracy on the FaceScape dataset within dozens of seconds.
Recovering detailed facial geometry from a set of calibrated multi-view images is valuable for its wide range of applications. Traditional multi-view stereo (MVS) methods adopt an optimization-based scheme to regularize the matching cost. Recently, learning-based methods integrate all these into an end-to-end neural network and show superiority of efficiency. In this paper, we propose a novel architecture to recover extremely detailed 3D faces within dozens of seconds. Unlike previous learning-based methods that regularize the cost volume via 3D CNN, we propose to learn an implicit function for regressing the matching cost. By fitting a 3D morphable model from multi-view images, the features of multiple images are extracted and aggregated in the mesh-attached UV space, which makes the implicit function more effective in recovering detailed facial shape. Our method outperforms SOTA learning-based MVS in accuracy by a large margin on the FaceScape dataset. The code and data are released in https://github.com/zhuhao-nju/mvfr.