Articulated motion discovery using pairs of trajectories
This work addresses the problem of unsupervised behavior segmentation in unconstrained animal videos for computer vision applications, representing an incremental improvement over existing trajectory-based methods.
The paper tackles the problem of discovering characteristic motion patterns in videos of highly articulated objects, such as tigers, by proposing an unsupervised approach that analyzes pairs of trajectories. The method outperforms existing descriptors like HOG and DTFs on datasets of dogs and tigers, enabling segmentation of videos into intervals containing single behaviors.
We propose an unsupervised approach for discovering characteristic motion patterns in videos of highly articulated objects performing natural, unscripted behaviors, such as tigers in the wild. We discover consistent patterns in a bottom-up manner by analyzing the relative displacements of large numbers of ordered trajectory pairs through time, such that each trajectory is attached to a different moving part on the object. The pairs of trajectories descriptor relies entirely on motion and is more discriminative than state-of-the-art features that employ single trajectories. Our method generates temporal video intervals, each automatically trimmed to one instance of the discovered behavior, and clusters them by type (e.g., running, turning head, drinking water). We present experiments on two datasets: dogs from YouTube-Objects and a new dataset of National Geographic tiger videos. Results confirm that our proposed descriptor outperforms existing appearance- and trajectory-based descriptors (e.g., HOG and DTFs) on both datasets and enables us to segment unconstrained animal video into intervals containing single behaviors.