Automated Audio Captioning with Epochal Difficult Captions for Curriculum Learning
This work addresses automated audio captioning, a domain-specific task, with an incremental improvement in training methodology.
The paper tackles the problem of automated audio captioning by proposing Epochal Difficult Captions, a curriculum learning algorithm that modifies target captions based on epoch-based difficulty, and shows consistent performance improvements across three systems.
In this paper, we propose an algorithm, Epochal Difficult Captions, to supplement the training of any model for the Automated Audio Captioning task. Epochal Difficult Captions is an elegant evolution to the keyword estimation task that previous work have used to train the encoder of the AAC model. Epochal Difficult Captions modifies the target captions based on a curriculum and a difficulty level determined as a function of current epoch. Epochal Difficult Captions can be used with any model architecture and is a lightweight function that does not increase training time. We test our results on three systems and show that using Epochal Difficult Captions consistently improves performance