FACTS: Facial Animation Creation using the Transfer of Styles
This addresses the need for cost-effective and expressive character animation in video games and entertainment, though it is incremental as it builds on existing audio-driven models and style transfer techniques.
The paper tackles the problem of generating expressive facial animations by enabling style transfer on existing 3D animations, using a StarGAN with a viseme-preserving loss to maintain lip-sync while modifying emotions and person-specific styles.
The ability to accurately capture and express emotions is a critical aspect of creating believable characters in video games and other forms of entertainment. Traditionally, this animation has been achieved with artistic effort or performance capture, both requiring costs in time and labor. More recently, audio-driven models have seen success, however, these often lack expressiveness in areas not correlated to the audio signal. In this paper, we present a novel approach to facial animation by taking existing animations and allowing for the modification of style characteristics. Specifically, we explore the use of a StarGAN to enable the conversion of 3D facial animations into different emotions and person-specific styles. We are able to maintain the lip-sync of the animations with this method thanks to the use of a novel viseme-preserving loss.