View-invariant Deep Architecture for Human Action Recognition using late fusion
This work addresses the challenge of view-invariant action recognition for applications like surveillance and human-computer interaction, presenting a novel integration of cues but with incremental methodological improvements.
The paper tackled the problem of human action recognition from unknown views by proposing a view-invariant deep framework that integrates motion and shape temporal dynamics cues, achieving significant outperformance over state-of-the-art methods on three public datasets in terms of accuracy, ROC, and AUC.
Human action Recognition for unknown views is a challenging task. We propose a view-invariant deep human action recognition framework, which is a novel integration of two important action cues: motion and shape temporal dynamics (STD). The motion stream encapsulates the motion content of action as RGB Dynamic Images (RGB-DIs) which are processed by the fine-tuned InceptionV3 model. The STD stream learns long-term view-invariant shape dynamics of action using human pose model (HPM) based view-invariant features mined from structural similarity index matrix (SSIM) based key depth human pose frames. To predict the score of the test sample, three types of late fusion (maximum, average and product) techniques are applied on individual stream scores. To validate the performance of the proposed novel framework the experiments are performed using both cross subject and cross-view validation schemes on three publically available benchmarks- NUCLA multi-view dataset, UWA3D-II Activity dataset and NTU RGB-D Activity dataset. Our algorithm outperforms with existing state-of-the-arts significantly that is reported in terms of accuracy, receiver operating characteristic (ROC) curve and area under the curve (AUC).