SoGAR: Self-supervised Spatiotemporal Attention-based Social Group Activity Recognition
This addresses group activity recognition in videos, which is important for surveillance and sports analysis, but appears incremental as it builds on existing transformer and self-supervised methods.
The paper tackles Social Group Activity Recognition by introducing a self-supervised transformer approach that uses unlabeled video data with local and global views at varying frame rates, achieving state-of-the-art results on three benchmarks (JRDB-PAR, NBA, Volleyball datasets) with improvements in F1-score, MCA, and MPCA metrics.
This paper introduces a novel approach to Social Group Activity Recognition (SoGAR) using Self-supervised Transformers network that can effectively utilize unlabeled video data. To extract spatio-temporal information, we created local and global views with varying frame rates. Our self-supervised objective ensures that features extracted from contrasting views of the same video were consistent across spatio-temporal domains. Our proposed approach is efficient in using transformer-based encoders to alleviate the weakly supervised setting of group activity recognition. By leveraging the benefits of transformer models, our approach can model long-term relationships along spatio-temporal dimensions. Our proposed SoGAR method achieved state-of-the-art results on three group activity recognition benchmarks, namely JRDB-PAR, NBA, and Volleyball datasets, surpassing the current numbers in terms of F1-score, MCA, and MPCA metrics.