Long-Short Temporal Modeling for Efficient Action Recognition
This work addresses action recognition, a key task in computer vision, but is incremental as it builds on existing two-stream and motion representation approaches.
The authors tackled the problem of efficient long-short temporal modeling for action recognition by proposing MENet, a two-stream network with Motion Enhancement and Video-level Aggregation modules, achieving improved performance on UCF101 and HMDB51 benchmarks.
Efficient long-short temporal modeling is key for enhancing the performance of action recognition task. In this paper, we propose a new two-stream action recognition network, termed as MENet, consisting of a Motion Enhancement (ME) module and a Video-level Aggregation (VLA) module to achieve long-short temporal modeling. Specifically, motion representations have been proved effective in capturing short-term and high-frequency action. However, current motion representations are calculated from adjacent frames, which may have poor interpretation and bring useless information (noisy or blank). Thus, for short-term motions, we design an efficient ME module to enhance the short-term motions by mingling the motion saliency among neighboring segments. As for long-term aggregations, VLA is adopted at the top of the appearance branch to integrate the long-term dependencies across all segments. The two components of MENet are complementary in temporal modeling. Extensive experiments are conducted on UCF101 and HMDB51 benchmarks, which verify the effectiveness and efficiency of our proposed MENet.