Enhancing Transformer for Video Understanding Using Gated Multi-Level Attention and Temporal Adversarial Training
This addresses video understanding for AI applications, presenting an incremental improvement by adapting Transformer models to videos with new techniques.
The paper tackles video understanding by introducing Gated Adversarial Transformer (GAT), which uses multi-level attention and adversarial training to improve performance on video categorization, achieving results on the YouTube-8M dataset.
The introduction of Transformer model has led to tremendous advancements in sequence modeling, especially in text domain. However, the use of attention-based models for video understanding is still relatively unexplored. In this paper, we introduce Gated Adversarial Transformer (GAT) to enhance the applicability of attention-based models to videos. GAT uses a multi-level attention gate to model the relevance of a frame based on local and global contexts. This enables the model to understand the video at various granularities. Further, GAT uses adversarial training to improve model generalization. We propose temporal attention regularization scheme to improve the robustness of attention modules to adversarial examples. We illustrate the performance of GAT on the large-scale YoutTube-8M data set on the task of video categorization. We further show ablation studies along with quantitative and qualitative analysis to showcase the improvement.