Co-Speech Gesture and Facial Expression Generation for Non-Photorealistic 3D Characters
It addresses the lack of suitable methods for non-photorealistic characters in conversational AI, which is an incremental advancement for animation and gaming applications.
This study tackled the problem of generating co-speech gestures and facial expressions for non-photorealistic 3D characters, such as anime, by using expression data from comics and dialogue-specific semantic gestures, resulting in significant improvements demonstrated in a user study.
With the advancement of conversational AI, research on bodily expressions, including gestures and facial expressions, has also progressed. However, many existing studies focus on photorealistic avatars, making them unsuitable for non-photorealistic characters, such as those found in anime. This study proposes methods for expressing emotions, including exaggerated expressions unique to non-photorealistic characters, by utilizing expression data extracted from comics and dialogue-specific semantic gestures. A user study demonstrated significant improvements across multiple aspects when compared to existing research.