Exploiting Temporal Coherence for Multi-modal Video Categorization
This addresses video content analysis for applications like object detection and scene understanding, but appears incremental as it builds on existing multimodal methods.
The paper tackled video categorization by developing a temporal coherence-based regularization approach for multimodal models, demonstrating that it outperforms strong state-of-the-art baselines in experiments.
Multimodal ML models can process data in multiple modalities (e.g., video, images, audio, text) and are useful for video content analysis in a variety of problems (e.g., object detection, scene understanding). In this paper, we focus on the problem of video categorization by using a multimodal approach. We have developed a novel temporal coherence-based regularization approach, which applies to different types of models (e.g., RNN, NetVLAD, Transformer). We demonstrate through experiments how our proposed multimodal video categorization models with temporal coherence out-perform strong state-of-the-art baseline models.