Aggregating Frame-level Features for Large-Scale Video Classification
This work addresses video understanding for multi-label classification in a competition setting, representing an incremental improvement over existing methods.
The paper tackles large-scale video classification on the YouTube-8M dataset by aggregating frame-level features using RNNs and generalized VLAD, achieving a GAP@20 score of 0.84198 on public test data and ranking 4th out of 650 teams.
This paper introduces the system we developed for the Google Cloud & YouTube-8M Video Understanding Challenge, which can be considered as a multi-label classification problem defined on top of the large scale YouTube-8M Dataset. We employ a large set of techniques to aggregate the provided frame-level feature representations and generate video-level predictions, including several variants of recurrent neural networks (RNN) and generalized VLAD. We also adopt several fusion strategies to explore the complementarity among the models. In terms of the official metric GAP@20 (global average precision at 20), our best fusion model attains 0.84198 on the public 50\% of test data and 0.84193 on the private 50\% of test data, ranking 4th out of 650 teams worldwide in the competition.