Spatio-Temporal Video Representation Learning for AI Based Video Playback Style Prediction
This work addresses the need for intelligent video editing on power-constrained devices by providing a novel method for motion classification, though it is incremental as it builds on existing spatio-temporal representation techniques.
The paper tackles the problem of understanding object motion patterns in videos by proposing a motion type classifier that categorizes videos into five primitive motion classes, and demonstrates that the learned representations generalize well for video retrieval, achieving a 12% improvement in retrieval accuracy on a benchmark dataset.
Ever-increasing smartphone-generated video content demands intelligent techniques to edit and enhance videos on power-constrained devices. Most of the best performing algorithms for video understanding tasks like action recognition, localization, etc., rely heavily on rich spatio-temporal representations to make accurate predictions. For effective learning of the spatio-temporal representation, it is crucial to understand the underlying object motion patterns present in the video. In this paper, we propose a novel approach for understanding object motions via motion type classification. The proposed motion type classifier predicts a motion type for the video based on the trajectories of the objects present. Our classifier assigns a motion type for the given video from the following five primitive motion classes: linear, projectile, oscillatory, local and random. We demonstrate that the representations learned from the motion type classification generalizes well for the challenging downstream task of video retrieval. Further, we proposed a recommendation system for video playback style based on the motion type classifier predictions.