Noise-to-Notes: Diffusion-based Generation and Refinement for Automatic Drum Transcription
This work addresses automatic drum transcription for music analysis applications, representing an incremental advance through a novel generative formulation.
The paper tackles automatic drum transcription by redefining it as a conditional generative task using a diffusion-based framework called Noise-to-Notes, which transforms audio-conditioned noise into drum events with velocities. The approach establishes new state-of-the-art performance across multiple benchmarks by incorporating features from music foundation models to enhance robustness.
Automatic drum transcription (ADT) is traditionally formulated as a discriminative task to predict drum events from audio spectrograms. In this work, we redefine ADT as a conditional generative task and introduce Noise-to-Notes (N2N), a framework leveraging diffusion modeling to transform audio-conditioned Gaussian noise into drum events with associated velocities. This generative diffusion approach offers distinct advantages, including a flexible speed-accuracy trade-off and strong inpainting capabilities. However, the generation of binary onset and continuous velocity values presents a challenge for diffusion models, and to overcome this, we introduce an Annealed Pseudo-Huber loss to facilitate effective joint optimization. Finally, to augment low-level spectrogram features, we propose incorporating features extracted from music foundation models (MFMs), which capture high-level semantic information and enhance robustness to out-of-domain drum audio. Experimental results demonstrate that including MFM features significantly improves robustness and N2N establishes a new state-of-the-art performance across multiple ADT benchmarks.