Action-Conditioned 3D Human Motion Synthesis with Transformer VAE
This work addresses the challenge of synthesizing human motion for applications such as action recognition and motion denoising, though it appears incremental as it builds on existing generative models and datasets.
The paper tackles the problem of generating realistic and diverse 3D human motion sequences conditioned on categorical actions, without requiring an initial pose, and shows improvements over state-of-the-art methods on datasets like NTU RGB+D, HumanAct12, and UESTC.
We tackle the problem of action-conditioned generation of realistic and diverse human motion sequences. In contrast to methods that complete, or extend, motion sequences, this task does not require an initial pose or sequence. Here we learn an action-aware latent representation for human motions by training a generative variational autoencoder (VAE). By sampling from this latent space and querying a certain duration through a series of positional encodings, we synthesize variable-length motion sequences conditioned on a categorical action. Specifically, we design a Transformer-based architecture, ACTOR, for encoding and decoding a sequence of parametric SMPL human body models estimated from action recognition datasets. We evaluate our approach on the NTU RGB+D, HumanAct12 and UESTC datasets and show improvements over the state of the art. Furthermore, we present two use cases: improving action recognition through adding our synthesized data to training, and motion denoising. Code and models are available on our project page.