Audio2Face-3D: Audio-driven Realistic Facial Animation For Digital Avatars
It addresses the need for efficient facial animation authoring for game characters and digital avatars, though it appears incremental as it builds on existing audio-driven animation concepts.
The paper tackles the problem of generating realistic facial animations for digital avatars from audio input, resulting in a system that enables real-time interaction and has been open-sourced for use by creators and developers.
Audio-driven facial animation presents an effective solution for animating digital avatars. In this paper, we detail the technical aspects of NVIDIA Audio2Face-3D, including data acquisition, network architecture, retargeting methodology, evaluation metrics, and use cases. Audio2Face-3D system enables real-time interaction between human users and interactive avatars, facilitating facial animation authoring for game characters. To assist digital avatar creators and game developers in generating realistic facial animations, we have open-sourced Audio2Face-3D networks, SDK, training framework, and example dataset.