Disjoint Pose and Shape for 3D Face Reconstruction
This addresses the challenge of noisy and stretched-out reconstructions in 3D face modeling from limited views, offering a more robust solution for applications like computer vision and graphics.
The paper tackles the problem of 3D face reconstruction from only two casually captured images by proposing an end-to-end pipeline that disjointly solves for pose and shape, achieving stable and accurate optimization with remarkable improvement over state-of-the-art methods in quantitative and qualitative results.
Existing methods for 3D face reconstruction from a few casually captured images employ deep learning based models along with a 3D Morphable Model(3DMM) as face geometry prior. Structure From Motion(SFM), followed by Multi-View Stereo (MVS), on the other hand, uses dozens of high-resolution images to reconstruct accurate 3D faces.However, it produces noisy and stretched-out results with only two views available. In this paper, taking inspiration from both these methods, we propose an end-to-end pipeline that disjointly solves for pose and shape to make the optimization stable and accurate. We use a face shape prior to estimate face pose and use stereo matching followed by a 3DMM to solve for the shape. The proposed method achieves end-to-end topological consistency, enables iterative face pose refinement procedure, and show remarkable improvement on both quantitative and qualitative results over existing state-of-the-art methods.