Animating the Uncaptured: Humanoid Mesh Animation with Video Diffusion Models
This provides a more accessible way for graphics applications to generate humanoid animations, though it appears incremental as it builds on existing video models and SMPL representations.
The paper tackles the problem of creating realistic animations for static 3D humanoid meshes, which is typically time-consuming and costly, by proposing a method that uses video diffusion models to synthesize 4D animated sequences from text prompts and rendered images, resulting in a cost-effective solution for diverse animations.
Animation of humanoid characters is essential in various graphics applications, but requires significant time and cost to create realistic animations. We propose an approach to synthesize 4D animated sequences of input static 3D humanoid meshes, leveraging strong generalized motion priors from generative video models -- as such video models contain powerful motion information covering a wide variety of human motions. From an input static 3D humanoid mesh and a text prompt describing the desired animation, we synthesize a corresponding video conditioned on a rendered image of the 3D mesh. We then employ an underlying SMPL representation to animate the corresponding 3D mesh according to the video-generated motion, based on our motion optimization. This enables a cost-effective and accessible solution to enable the synthesis of diverse and realistic 4D animations.