CLearViD: Curriculum Learning for Video Description
This addresses the problem of generating coherent video descriptions for AI applications, but it is incremental as it builds on existing transformer and curriculum learning methods.
The paper tackles video description generation by introducing CLearViD, a transformer-based model that uses curriculum learning with noise and dropout strategies, and it significantly outperforms state-of-the-art models on ActivityNet Captions and YouCook2 datasets in accuracy and diversity metrics.
Video description entails automatically generating coherent natural language sentences that narrate the content of a given video. We introduce CLearViD, a transformer-based model for video description generation that leverages curriculum learning to accomplish this task. In particular, we investigate two curriculum strategies: (1) progressively exposing the model to more challenging samples by gradually applying a Gaussian noise to the video data, and (2) gradually reducing the capacity of the network through dropout during the training process. These methods enable the model to learn more robust and generalizable features. Moreover, CLearViD leverages the Mish activation function, which provides non-linearity and non-monotonicity and helps alleviate the issue of vanishing gradients. Our extensive experiments and ablation studies demonstrate the effectiveness of the proposed model. The results on two datasets, namely ActivityNet Captions and YouCook2, show that CLearViD significantly outperforms existing state-of-the-art models in terms of both accuracy and diversity metrics.