Self-supervised Multi-actor Social Activity Understanding in Streaming Videos
This addresses the problem of understanding social interactions in videos for applications like surveillance and assistive robotics, offering a privacy-friendly alternative to heavily annotated methods, though it appears incremental as it builds on existing benchmarks.
The paper tackles Social Activity Recognition (SAR) in streaming videos by proposing a self-supervised multi-actor predictive learning approach using a visual-semantic graph structure, achieving competitive performance on group activity recognition benchmarks and demonstrating generalizability to arbitrary action localization on three public benchmarks.
This work addresses the problem of Social Activity Recognition (SAR), a critical component in real-world tasks like surveillance and assistive robotics. Unlike traditional event understanding approaches, SAR necessitates modeling individual actors' appearance and motions and contextualizing them within their social interactions. Traditional action localization methods fall short due to their single-actor, single-action assumption. Previous SAR research has relied heavily on densely annotated data, but privacy concerns limit their applicability in real-world settings. In this work, we propose a self-supervised approach based on multi-actor predictive learning for SAR in streaming videos. Using a visual-semantic graph structure, we model social interactions, enabling relational reasoning for robust performance with minimal labeled data. The proposed framework achieves competitive performance on standard group activity recognition benchmarks. Evaluation on three publicly available action localization benchmarks demonstrates its generalizability to arbitrary action localization.