Co-occurrence Feature Learning for Skeleton based Action Recognition using Regularized Deep LSTM Networks
This work addresses action recognition for computer vision applications, but it is incremental as it builds on existing LSTM methods with specific enhancements.
The paper tackles skeleton-based action recognition by proposing a deep LSTM network with a novel regularization scheme to learn co-occurrence features of joints, achieving effective results on three datasets.
Skeleton based action recognition distinguishes human actions using the trajectories of skeleton joints, which provide a very good representation for describing actions. Considering that recurrent neural networks (RNNs) with Long Short-Term Memory (LSTM) can learn feature representations and model long-term temporal dependencies automatically, we propose an end-to-end fully connected deep LSTM network for skeleton based action recognition. Inspired by the observation that the co-occurrences of the joints intrinsically characterize human actions, we take the skeleton as the input at each time slot and introduce a novel regularization scheme to learn the co-occurrence features of skeleton joints. To train the deep LSTM network effectively, we propose a new dropout algorithm which simultaneously operates on the gates, cells, and output responses of the LSTM neurons. Experimental results on three human action recognition datasets consistently demonstrate the effectiveness of the proposed model.