FaceLift: Learning Generalizable Single Image 3D Face Reconstruction from Synthetic Heads
This addresses the challenge of full-view 3D face reconstruction from monocular images for applications in computer vision and graphics, representing a strong specific gain rather than a foundational breakthrough.
The paper tackles the problem of reconstructing 3D faces from a single image by introducing FaceLift, which uses a multi-view latent diffusion model and transformer-based reconstructor to achieve high-quality 360-degree 3D head reconstruction, outperforming state-of-the-art methods on identity preservation, detail recovery, and rendering quality.
We present FaceLift, a novel feed-forward approach for generalizable high-quality 360-degree 3D head reconstruction from a single image. Our pipeline first employs a multi-view latent diffusion model to generate consistent side and back views from a single facial input, which then feeds into a transformer-based reconstructor that produces a comprehensive 3D Gaussian splats representation. Previous methods for monocular 3D face reconstruction often lack full view coverage or view consistency due to insufficient multi-view supervision. We address this by creating a high-quality synthetic head dataset that enables consistent supervision across viewpoints. To bridge the domain gap between synthetic training data and real-world images, we propose a simple yet effective technique that ensures the view generation process maintains fidelity to the input by learning to reconstruct the input image alongside the view generation. Despite being trained exclusively on synthetic data, our method demonstrates remarkable generalization to real-world images. Through extensive qualitative and quantitative evaluations, we show that FaceLift outperforms state-of-the-art 3D face reconstruction methods on identity preservation, detail recovery, and rendering quality.