Three-stream network for enriched Action Recognition
This work addresses the challenge of accurately recognizing human activities in videos, which is important for machine intelligence applications, but it appears incremental as it builds on existing multi-stream CNN approaches.
The paper tackled the problem of human action recognition in videos by proposing two CNN-based architectures with three streams operating at different frame rates to capture spatial and temporal information, achieving state-of-the-art performance on UCF-101, Kinetics-600, and AVA datasets.
Understanding accurate information on human behaviours is one of the most important tasks in machine intelligence. Human Activity Recognition that aims to understand human activities from a video is a challenging task due to various problems including background, camera motion and dataset variations. This paper proposes two CNN based architectures with three streams which allow the model to exploit the dataset under different settings. The three pathways are differentiated in frame rates. The single pathway, operates at a single frame rate captures spatial information, the slow pathway operates at low frame rates captures the spatial information and the fast pathway operates at high frame rates that capture fine temporal information. Post CNN encoders, we add bidirectional LSTM and attention heads respectively to capture the context and temporal features. By experimenting with various algorithms on UCF-101, Kinetics-600 and AVA dataset, we observe that the proposed models achieve state-of-art performance for human action recognition task.