ReconVAT: A Semi-Supervised Automatic Music Transcription Framework for Low-Resource Real-World Data
This addresses the generalization issue in automatic music transcription for low-resource real-world data, offering an incremental improvement through semi-supervised learning.
The paper tackles the problem of automatic music transcription models lacking generalization to real-world music from diverse genres not in labeled training data, proposing ReconVAT, a semi-supervised framework that uses reconstruction loss and virtual adversarial training to achieve competitive results, such as F1-score improvements of 22.2% and 62.5% on MusicNet benchmarks.
Most of the current supervised automatic music transcription (AMT) models lack the ability to generalize. This means that they have trouble transcribing real-world music recordings from diverse musical genres that are not presented in the labelled training data. In this paper, we propose a semi-supervised framework, ReconVAT, which solves this issue by leveraging the huge amount of available unlabelled music recordings. The proposed ReconVAT uses reconstruction loss and virtual adversarial training. When combined with existing U-net models for AMT, ReconVAT achieves competitive results on common benchmark datasets such as MAPS and MusicNet. For example, in the few-shot setting for the string part version of MusicNet, ReconVAT achieves F1-scores of 61.0% and 41.6% for the note-wise and note-with-offset-wise metrics respectively, which translates into an improvement of 22.2% and 62.5% compared to the supervised baseline model. Our proposed framework also demonstrates the potential of continual learning on new data, which could be useful in real-world applications whereby new data is constantly available.