CASTELLA: Long Audio Dataset with Captions and Temporal Boundaries
This provides a real-world dataset for audio moment retrieval, addressing a gap for researchers and practitioners, though it is incremental as it builds on existing synthetic data approaches.
The authors tackled the lack of a reliable benchmark for audio moment retrieval (AMR) by introducing CASTELLA, a large-scale manually annotated dataset with 1,862 audio recordings, which is 24 times larger than previous datasets, and they showed that fine-tuning on it improved performance by 10.4 points in Recall1@0.7 compared to using synthetic data alone.
We introduce CASTELLA, a human-annotated audio benchmark for the task of audio moment retrieval (AMR). Although AMR has various useful potential applications, there is still no established benchmark with real-world data. The early study of AMR trained the model with solely synthetic datasets. Moreover, the evaluation is based on annotated dataset of fewer than 100 samples. This resulted in less reliable reported performance. To ensure performance for applications in real-world environments, we present CASTELLA, a large-scale manually annotated AMR dataset. CASTELLA consists of 1,009, 213, and 640 audio recordings for train, valid, and test split, respectively, which is 24 times larger than the previous dataset. We also establish a baseline model for AMR using CASTELLA. Our experiments demonstrate that a model fine-tuned on CASTELLA after pre-training on the synthetic data outperformed a model trained solely on the synthetic data by 10.4 points in Recall1@0.7. CASTELLA is publicly available in https://h-munakata.github.io/CASTELLA-demo/.