SplatFace: Gaussian Splat Face Reconstruction Leveraging an Optimizable Surface
This work addresses face reconstruction for applications like graphics and VR, but it is incremental as it builds on existing Gaussian splatting and 3DMM techniques.
The authors tackled 3D human face reconstruction from limited images by developing SplatFace, a Gaussian splatting framework that integrates a 3D Morphable Model for surface guidance, resulting in competitive performance in both novel view synthesis and accurate 3D mesh generation.
We present SplatFace, a novel Gaussian splatting framework designed for 3D human face reconstruction without reliance on accurate pre-determined geometry. Our method is designed to simultaneously deliver both high-quality novel view rendering and accurate 3D mesh reconstructions. We incorporate a generic 3D Morphable Model (3DMM) to provide a surface geometric structure, making it possible to reconstruct faces with a limited set of input images. We introduce a joint optimization strategy that refines both the Gaussians and the morphable surface through a synergistic non-rigid alignment process. A novel distance metric, splat-to-surface, is proposed to improve alignment by considering both the Gaussian position and covariance. The surface information is also utilized to incorporate a world-space densification process, resulting in superior reconstruction quality. Our experimental analysis demonstrates that the proposed method is competitive with both other Gaussian splatting techniques in novel view synthesis and other 3D reconstruction methods in producing 3D face meshes with high geometric precision.