CSTA: CNN-based Spatiotemporal Attention for Video Summarization
This work addresses video summarization for efficient content analysis, but it is incremental as it builds on existing attention mechanisms with a novel CNN-based approach.
The authors tackled the problem of video summarization by proposing a CNN-based Spatiotemporal Attention (CSTA) method that uses 2D CNN on frame features to capture visual significance, achieving state-of-the-art performance on SumMe and TVSum datasets with fewer MACs.
Video summarization aims to generate a concise representation of a video, capturing its essential content and key moments while reducing its overall length. Although several methods employ attention mechanisms to handle long-term dependencies, they often fail to capture the visual significance inherent in frames. To address this limitation, we propose a CNN-based SpatioTemporal Attention (CSTA) method that stacks each feature of frames from a single video to form image-like frame representations and applies 2D CNN to these frame features. Our methodology relies on CNN to comprehend the inter and intra-frame relations and to find crucial attributes in videos by exploiting its ability to learn absolute positions within images. In contrast to previous work compromising efficiency by designing additional modules to focus on spatial importance, CSTA requires minimal computational overhead as it uses CNN as a sliding window. Extensive experiments on two benchmark datasets (SumMe and TVSum) demonstrate that our proposed approach achieves state-of-the-art performance with fewer MACs compared to previous methods. Codes are available at https://github.com/thswodnjs3/CSTA.