S3Aug: Segmentation, Sampling, and Shift for Action Recognition
This addresses the need for better data augmentation in action recognition, especially for handling out-of-context videos, but it is incremental as it builds on existing video augmentation techniques.
The paper tackles the problem of generating diverse training videos for action recognition by proposing S3Aug, a data augmentation method that creates new videos from a single source through segmentation, label-to-image transformation, sampling, and feature shifting, achieving improved performance on datasets like UCF101, HMDB51, and Mimetics, with particular effectiveness on out-of-context videos.
Action recognition is a well-established area of research in computer vision. In this paper, we propose S3Aug, a video data augmenatation for action recognition. Unlike conventional video data augmentation methods that involve cutting and pasting regions from two videos, the proposed method generates new videos from a single training video through segmentation and label-to-image transformation. Furthermore, the proposed method modifies certain categories of label images by sampling to generate a variety of videos, and shifts intermediate features to enhance the temporal coherency between frames of the generate videos. Experimental results on the UCF101, HMDB51, and Mimetics datasets demonstrate the effectiveness of the proposed method, paricularlly for out-of-context videos of the Mimetics dataset.