Triangular Character Animation Sampling with Motion, Emotion, and Relation
This work addresses the challenge of synthesizing interactions between characters in 3D animations, which is incremental as it builds on existing individual character animation techniques to enable automatic generation for animators and NPCs in VR.
The paper tackles the problem of automatically controlling interactions between characters in animations by proposing an energy-based framework that samples animations based on body motions, facial expressions, and social relations. The method uses a Spatial-Temporal And-Or graph to encode these relationships and demonstrates the ability to recognize social relations and generate new scenes with vivid motion and emotion using MCMC.
Dramatic progress has been made in animating individual characters. However, we still lack automatic control over activities between characters, especially those involving interactions. In this paper, we present a novel energy-based framework to sample and synthesize animations by associating the characters' body motions, facial expressions, and social relations. We propose a Spatial-Temporal And-Or graph (ST-AOG), a stochastic grammar model, to encode the contextual relationship between motion, emotion, and relation, forming a triangle in a conditional random field. We train our model from a labeled dataset of two-character interactions. Experiments demonstrate that our method can recognize the social relation between two characters and sample new scenes of vivid motion and emotion using Markov Chain Monte Carlo (MCMC) given the social relation. Thus, our method can provide animators with an automatic way to generate 3D character animations, help synthesize interactions between Non-Player Characters (NPCs), and enhance machine emotion intelligence (EQ) in virtual reality (VR).