Spatio-Temporal LSTM with Trust Gates for 3D Human Action Recognition
This addresses the problem of robust action recognition from noisy 3D skeleton data for applications like surveillance or human-computer interaction, with incremental improvements over existing methods.
The paper tackles 3D human action recognition by extending RNN-based methods to spatio-temporal domains and introducing a tree-structure traversal and a new gating mechanism in LSTM to handle noise and occlusion, achieving state-of-the-art performance on 4 benchmark datasets.
3D action recognition - analysis of human actions based on 3D skeleton data - becomes popular recently due to its succinctness, robustness, and view-invariant representation. Recent attempts on this problem suggested to develop RNN-based learning methods to model the contextual dependency in the temporal domain. In this paper, we extend this idea to spatio-temporal domains to analyze the hidden sources of action-related information within the input data over both domains concurrently. Inspired by the graphical structure of the human skeleton, we further propose a more powerful tree-structure based traversal method. To handle the noise and occlusion in 3D skeleton data, we introduce new gating mechanism within LSTM to learn the reliability of the sequential input data and accordingly adjust its effect on updating the long-term context information stored in the memory cell. Our method achieves state-of-the-art performance on 4 challenging benchmark datasets for 3D human action analysis.