AU-Blendshape for Fine-grained Stylized 3D Facial Expression Manipulation
This addresses the problem of creating detailed, stylized 3D facial animations for computer graphics and animation professionals, though it appears incremental as it builds on existing AU-based methods with new data and a network.
The paper tackles the challenge of fine-grained stylized 3D facial expression manipulation by introducing AUBlendSet, a dataset with 500 identities based on 32 facial action units, and AUBlendNet, a network that predicts AU-blendshape basis vectors for different styles, achieving stylized emotional facial manipulation validated through tasks like speech-driven animation and emotion recognition augmentation.
While 3D facial animation has made impressive progress, challenges still exist in realizing fine-grained stylized 3D facial expression manipulation due to the lack of appropriate datasets. In this paper, we introduce the AUBlendSet, a 3D facial dataset based on AU-Blendshape representation for fine-grained facial expression manipulation across identities. AUBlendSet is a blendshape data collection based on 32 standard facial action units (AUs) across 500 identities, along with an additional set of facial postures annotated with detailed AUs. Based on AUBlendSet, we propose AUBlendNet to learn AU-Blendshape basis vectors for different character styles. AUBlendNet predicts, in parallel, the AU-Blendshape basis vectors of the corresponding style for a given identity mesh, thereby achieving stylized 3D emotional facial manipulation. We comprehensively validate the effectiveness of AUBlendSet and AUBlendNet through tasks such as stylized facial expression manipulation, speech-driven emotional facial animation, and emotion recognition data augmentation. Through a series of qualitative and quantitative experiments, we demonstrate the potential and importance of AUBlendSet and AUBlendNet in 3D facial animation tasks. To the best of our knowledge, AUBlendSet is the first dataset, and AUBlendNet is the first network for continuous 3D facial expression manipulation for any identity through facial AUs. Our source code is available at https://github.com/wslh852/AUBlendNet.git.