Co-Regularized Deep Representations for Video Summarization
This work addresses the challenge of creating relevant video summaries for video sharing platforms, offering an incremental improvement over existing methods.
The authors tackled the problem of generating compact and compelling video summaries by proposing a co-regularized deep learning framework that combines CNNs and RBMs to select keyframes, and their method consistently outperformed baseline schemes, including one from a popular video sharing website, in user studies on attractiveness and informativeness, with a more significant lead for smaller summaries.
Compact keyframe-based video summaries are a popular way of generating viewership on video sharing platforms. Yet, creating relevant and compelling summaries for arbitrarily long videos with a small number of keyframes is a challenging task. We propose a comprehensive keyframe-based summarization framework combining deep convolutional neural networks and restricted Boltzmann machines. An original co-regularization scheme is used to discover meaningful subject-scene associations. The resulting multimodal representations are then used to select highly-relevant keyframes. A comprehensive user study is conducted comparing our proposed method to a variety of schemes, including the summarization currently in use by one of the most popular video sharing websites. The results show that our method consistently outperforms the baseline schemes for any given amount of keyframes both in terms of attractiveness and informativeness. The lead is even more significant for smaller summaries.