Spatial-Temporal Alignment Network for Action Recognition
This work addresses viewpoint invariance for action recognition, offering a lightweight, plug-in solution that is incremental to existing architectures.
The paper tackles the challenge of geometric variations in action recognition by proposing a Spatial-Temporal Alignment Network (STAN) that learns viewpoint invariant features, which consistently improves state-of-the-art models on datasets like UCF101 and HMDB51.
This paper studies introducing viewpoint invariant feature representations in existing action recognition architecture. Despite significant progress in action recognition, efficiently handling geometric variations in large-scale datasets remains challenging. To tackle this problem, we propose a novel Spatial-Temporal Alignment Network (STAN), which explicitly learns geometric invariant representations for action recognition. Notably, the STAN model is light-weighted and generic, which could be plugged into existing action recognition models (e.g., MViTv2) with a low extra computational cost. We test our STAN model on widely-used datasets like UCF101 and HMDB51. The experimental results show that the STAN model can consistently improve the state-of-the-art models in action recognition tasks in trained-from-scratch settings.