Action Recognition using Visual Attention
This work addresses action recognition for video analysis applications, presenting an incremental improvement by applying attention mechanisms to existing RNN/LSTM frameworks.
The authors tackled action recognition in videos by proposing a soft attention model using multi-layered RNNs with LSTM units, which learns to selectively focus on relevant parts of video frames and classifies videos after a few glimpses, achieving evaluation on UCF-11, HMDB-51, and Hollywood2 datasets.
We propose a soft attention based model for the task of action recognition in videos. We use multi-layered Recurrent Neural Networks (RNNs) with Long Short-Term Memory (LSTM) units which are deep both spatially and temporally. Our model learns to focus selectively on parts of the video frames and classifies videos after taking a few glimpses. The model essentially learns which parts in the frames are relevant for the task at hand and attaches higher importance to them. We evaluate the model on UCF-11 (YouTube Action), HMDB-51 and Hollywood2 datasets and analyze how the model focuses its attention depending on the scene and the action being performed.