Multi-modal Aggregation for Video Classification
This work addresses video classification for researchers and practitioners, but it is incremental as it applies existing methods like 3D convolution and ensemble techniques to a specific dataset.
The paper tackled the Large-Scale Video Classification Challenge by aggregating visual, motion, and audio modalities, achieving first place with an mAP of 0.8741 on the testing set using an ensemble model.
In this paper, we present a solution to Large-Scale Video Classification Challenge (LSVC2017) [1] that ranked the 1st place. We focused on a variety of modalities that cover visual, motion and audio. Also, we visualized the aggregation process to better understand how each modality takes effect. Among the extracted modalities, we found Temporal-Spatial features calculated by 3D convolution quite promising that greatly improved the performance. We attained the official metric mAP 0.8741 on the testing set with the ensemble model.