Unaligned Supervision For Automatic Music Transcription in The Wild
This addresses the data collection bottleneck in AMT for various instruments, enabling broader application beyond piano and guitar, though it is an incremental improvement over existing alignment-based methods.
The paper tackles the problem of multi-instrument Automatic Music Transcription (AMT) by introducing NoteEM, a method that uses unaligned supervision to train on in-the-wild recordings, achieving state-of-the-art note-level accuracy on the MAPS dataset and strong cross-dataset performance.
Multi-instrument Automatic Music Transcription (AMT), or the decoding of a musical recording into semantic musical content, is one of the holy grails of Music Information Retrieval. Current AMT approaches are restricted to piano and (some) guitar recordings, due to difficult data collection. In order to overcome data collection barriers, previous AMT approaches attempt to employ musical scores in the form of a digitized version of the same song or piece. The scores are typically aligned using audio features and strenuous human intervention to generate training labels. We introduce NoteEM, a method for simultaneously training a transcriber and aligning the scores to their corresponding performances, in a fully-automated process. Using this unaligned supervision scheme, complemented by pseudo-labels and pitch-shift augmentation, our method enables training on in-the-wild recordings with unprecedented accuracy and instrumental variety. Using only synthetic data and unaligned supervision, we report SOTA note-level accuracy of the MAPS dataset, and large favorable margins on cross-dataset evaluations. We also demonstrate robustness and ease of use; we report comparable results when training on a small, easily obtainable, self-collected dataset, and we propose alternative labeling to the MusicNet dataset, which we show to be more accurate. Our project page is available at https://benadar293.github.io