Multi-NeuS: 3D Head Portraits from Single Image with Neural Implicit Functions
This addresses the challenge of few-shot 3D head reconstruction for applications like virtual avatars, though it is incremental as it builds on existing neural implicit methods.
The paper tackles the problem of reconstructing textured 3D meshes of human heads from one or few views by extending NeuS, a neural implicit function formulation, to learn priors from data and generalize to unseen heads, achieving good results with training on just a hundred smartphone videos.
We present an approach for the reconstruction of textured 3D meshes of human heads from one or few views. Since such few-shot reconstruction is underconstrained, it requires prior knowledge which is hard to impose on traditional 3D reconstruction algorithms. In this work, we rely on the recently introduced 3D representation $\unicode{x2013}$ neural implicit functions $\unicode{x2013}$ which, being based on neural networks, allows to naturally learn priors about human heads from data, and is directly convertible to textured mesh. Namely, we extend NeuS, a state-of-the-art neural implicit function formulation, to represent multiple objects of a class (human heads in our case) simultaneously. The underlying neural net architecture is designed to learn the commonalities among these objects and to generalize to unseen ones. Our model is trained on just a hundred smartphone videos and does not require any scanned 3D data. Afterwards, the model can fit novel heads in the few-shot or one-shot modes with good results.