HAvatar: High-fidelity Head Avatar via Facial Model Conditioned Neural Radiance Field
This addresses the need for high-fidelity, animatable head avatars in applications like virtual reality or gaming, but it appears incremental as it builds on existing NeRF and parametric model techniques.
The paper tackles the problem of creating animatable 3D human head avatars that balance realism and expression control by introducing a hybrid explicit-implicit representation, achieving state-of-the-art performance in 3D head avatar animation.
The problem of modeling an animatable 3D human head avatar under light-weight setups is of significant importance but has not been well solved. Existing 3D representations either perform well in the realism of portrait images synthesis or the accuracy of expression control, but not both. To address the problem, we introduce a novel hybrid explicit-implicit 3D representation, Facial Model Conditioned Neural Radiance Field, which integrates the expressiveness of NeRF and the prior information from the parametric template. At the core of our representation, a synthetic-renderings-based condition method is proposed to fuse the prior information from the parametric model into the implicit field without constraining its topological flexibility. Besides, based on the hybrid representation, we properly overcome the inconsistent shape issue presented in existing methods and improve the animation stability. Moreover, by adopting an overall GAN-based architecture using an image-to-image translation network, we achieve high-resolution, realistic and view-consistent synthesis of dynamic head appearance. Experiments demonstrate that our method can achieve state-of-the-art performance for 3D head avatar animation compared with previous methods.