GaussianAvatars: Photorealistic Head Avatars with Rigged 3D Gaussians
This addresses the need for high-quality, controllable digital avatars in applications like virtual reality and film, representing a novel hybrid approach rather than a foundational breakthrough.
The paper tackles the problem of creating photorealistic head avatars with full control over expression, pose, and viewpoint, achieving significant improvements over existing methods in reenactment scenarios.
We introduce GaussianAvatars, a new method to create photorealistic head avatars that are fully controllable in terms of expression, pose, and viewpoint. The core idea is a dynamic 3D representation based on 3D Gaussian splats that are rigged to a parametric morphable face model. This combination facilitates photorealistic rendering while allowing for precise animation control via the underlying parametric model, e.g., through expression transfer from a driving sequence or by manually changing the morphable model parameters. We parameterize each splat by a local coordinate frame of a triangle and optimize for explicit displacement offset to obtain a more accurate geometric representation. During avatar reconstruction, we jointly optimize for the morphable model parameters and Gaussian splat parameters in an end-to-end fashion. We demonstrate the animation capabilities of our photorealistic avatar in several challenging scenarios. For instance, we show reenactments from a driving video, where our method outperforms existing works by a significant margin.