Sympathy for the Details: Dense Trajectories and Hybrid Classification Architectures for Action Recognition
This work addresses the problem of action recognition for video analysis, offering an incremental advance by hybridizing existing techniques to enhance data efficiency and performance.
The paper tackles action recognition in videos by introducing a hybrid model that combines unsupervised hand-crafted features with supervised deep networks, achieving significant improvements over state-of-the-art methods while being data-efficient, trained on 150 to 10,000 short clips.
Action recognition in videos is a challenging task due to the complexity of the spatio-temporal patterns to model and the difficulty to acquire and learn on large quantities of video data. Deep learning, although a breakthrough for image classification and showing promise for videos, has still not clearly superseded action recognition methods using hand-crafted features, even when training on massive datasets. In this paper, we introduce hybrid video classification architectures based on carefully designed unsupervised representations of hand-crafted spatio-temporal features classified by supervised deep networks. As we show in our experiments on five popular benchmarks for action recognition, our hybrid model combines the best of both worlds: it is data efficient (trained on 150 to 10000 short clips) and yet improves significantly on the state of the art, including recent deep models trained on millions of manually labelled images and videos.