Predicting Long-Term Skeletal Motions by a Spatio-Temporal Hierarchical Recurrent Network
This work addresses the problem of generating realistic long-term skeletal motions for applications like animation or robotics, representing an incremental improvement over prior methods.
The paper tackles the challenge of predicting long-term skeletal motions, where existing methods often produce unrealistic poses, by introducing a hierarchical recurrent network that models spatio-temporal dependencies and uses a Lie algebra latent representation. The approach significantly outperforms state-of-the-art methods on benchmark datasets for both short-term and long-term prediction.
The primary goal of skeletal motion prediction is to generate future motion by observing a sequence of 3D skeletons. A key challenge in motion prediction is the fact that a motion can often be performed in several different ways, with each consisting of its own configuration of poses and their spatio-temporal dependencies, and as a result, the predicted poses often converge to the motionless poses or non-human like motions in long-term prediction. This leads us to define a hierarchical recurrent network model that explicitly characterizes these internal configurations of poses and their local and global spatio-temporal dependencies. The model introduces a latent vector variable from the Lie algebra to represent spatial and temporal relations simultaneously. Furthermore, a structured stack LSTM-based decoder is devised to decode the predicted poses with a new loss function defined to estimate the quantized weight of each body part in a pose. Empirical evaluations on benchmark datasets suggest our approach significantly outperforms the state-of-the-art methods on both short-term and long-term motion prediction.