Self-supervised Learning for Unintentional Action Prediction
This work addresses the challenge of recognizing unintentional actions for intelligent systems in human environments, but it is incremental as it builds on prior self-supervised approaches by incorporating global context.
The paper tackles the problem of predicting unintentional actions in videos by developing a self-supervised representation learning method that uses global context, achieving improvements in classification, localization, and anticipation tasks.
Distinguishing if an action is performed as intended or if an intended action fails is an important skill that not only humans have, but that is also important for intelligent systems that operate in human environments. Recognizing if an action is unintentional or anticipating if an action will fail, however, is not straightforward due to lack of annotated data. While videos of unintentional or failed actions can be found in the Internet in abundance, high annotation costs are a major bottleneck for learning networks for these tasks. In this work, we thus study the problem of self-supervised representation learning for unintentional action prediction. While previous works learn the representation based on a local temporal neighborhood, we show that the global context of a video is needed to learn a good representation for the three downstream tasks: unintentional action classification, localization and anticipation. In the supplementary material, we show that the learned representation can be used for detecting anomalies in videos as well.