Leveraging Self-Supervised Training for Unintentional Action Recognition
This work addresses the challenge of rare and context-dependent unintentional actions for video analysis applications, representing an incremental advancement in action recognition.
The paper tackles the problem of recognizing unintentional actions in videos by identifying transitions from intentional to unintentional states, using a multi-stage framework with self-supervised training via temporal transformations, which significantly improves over state-of-the-art results.
Unintentional actions are rare occurrences that are difficult to define precisely and that are highly dependent on the temporal context of the action. In this work, we explore such actions and seek to identify the points in videos where the actions transition from intentional to unintentional. We propose a multi-stage framework that exploits inherent biases such as motion speed, motion direction, and order to recognize unintentional actions. To enhance representations via self-supervised training for the task of unintentional action recognition we propose temporal transformations, called Temporal Transformations of Inherent Biases of Unintentional Actions (T2IBUA). The multi-stage approach models the temporal information on both the level of individual frames and full clips. These enhanced representations show strong performance for unintentional action recognition tasks. We provide an extensive ablation study of our framework and report results that significantly improve over the state-of-the-art.