Piano Transcription by Hierarchical Language Modeling with Pretrained Roll-based Encoders
This work addresses challenges in music transcription for audio processing applications, offering an incremental improvement as a plug-in for existing encoders.
The paper tackles the problem of automatic music transcription by proposing a hybrid method that combines pre-trained roll-based encoders with a language model decoder and a hierarchical prediction strategy, resulting in improved onset-offset-velocity F1 scores of 0.01 and 0.022 over traditional methods on benchmark encoders.
Automatic Music Transcription (AMT), aiming to get musical notes from raw audio, typically uses frame-level systems with piano-roll outputs or language model (LM)-based systems with note-level predictions. However, frame-level systems require manual thresholding, while the LM-based systems struggle with long sequences. In this paper, we propose a hybrid method combining pre-trained roll-based encoders with an LM decoder to leverage the strengths of both methods. Besides, our approach employs a hierarchical prediction strategy, first predicting onset and pitch, then velocity, and finally offset. The hierarchical prediction strategy reduces computational costs by breaking down long sequences into different hierarchies. Evaluated on two benchmark roll-based encoders, our method outperforms traditional piano-roll outputs 0.01 and 0.022 in onset-offset-velocity F1 score, demonstrating its potential as a performance-enhancing plug-in for arbitrary roll-based music transcription encoder.