Exploring System Adaptations For Minimum Latency Real-Time Piano Transcription
This work addresses the need for low-latency transcription in real-time musical applications, but it is incremental as it adapts existing models rather than introducing a new paradigm.
The paper tackled the problem of adapting state-of-the-art online piano transcription models for real-time applications requiring latencies below 30 ms, finding that strictly causal processing reduces accuracy and there is a tradeoff between preprocessing latency and prediction accuracy.
Advances in neural network design and the availability of large-scale labeled datasets have driven major improvements in piano transcription. Existing approaches target either offline applications, with no restrictions on computational demands, or online transcription, with delays of 128-320 ms. However, most real-time musical applications require latencies below 30 ms. In this work, we investigate whether and how the current state-of-the-art online transcription model can be adapted for real-time piano transcription. Specifically, we eliminate all non-causal processing, and reduce computational load through shared computations across core model components and variations in model size. Additionally, we explore different pre- and postprocessing strategies, and related label encoding schemes, and discuss their suitability for real-time transcription. Evaluating the adaptions on the MAESTRO dataset, we find a drop in transcription accuracy due to strictly causal processing as well as a tradeoff between the preprocessing latency and prediction accuracy. We release our system as a baseline to support researchers in designing models towards minimum latency real-time transcription.