ImFace: A Nonlinear 3D Morphable Face Model with Implicit Neural Representations
This work addresses the problem of precise 3D face representation for computer vision and graphics applications, offering a novel approach with incremental improvements in modeling flexibility and detail.
The paper tackles the challenge of capturing accurate identity and expression details in 3D face models by introducing ImFace, a nonlinear 3D morphable face model using implicit neural representations, which achieves state-of-the-art performance with improved disentanglement and detail synthesis as shown in extensive experiments.
Precise representations of 3D faces are beneficial to various computer vision and graphics applications. Due to the data discretization and model linearity, however, it remains challenging to capture accurate identity and expression clues in current studies. This paper presents a novel 3D morphable face model, namely ImFace, to learn a nonlinear and continuous space with implicit neural representations. It builds two explicitly disentangled deformation fields to model complex shapes associated with identities and expressions, respectively, and designs an improved learning strategy to extend embeddings of expressions to allow more diverse changes. We further introduce a Neural Blend-Field to learn sophisticated details by adaptively blending a series of local fields. In addition to ImFace, an effective preprocessing pipeline is proposed to address the issue of watertight input requirement in implicit representations, enabling them to work with common facial surfaces for the first time. Extensive experiments are performed to demonstrate the superiority of ImFace.