Source Separation & Automatic Transcription for Music
This work addresses the problem of creating accessible sheet music from audio for musicians and audio producers, but it appears incremental as it builds on existing deep learning methods without claiming major breakthroughs.
The paper tackles the challenges of source separation and automatic music transcription by developing an end-to-end pipeline that uses spectrogram masking and deep neural networks to separate audio mixtures into instrument stems and transcribe them into sheet music, aiming to reduce audio noise and training times.
Source separation is the process of isolating individual sounds in an auditory mixture of multiple sounds [1], and has a variety of applications ranging from speech enhancement and lyric transcription [2] to digital audio production for music. Furthermore, Automatic Music Transcription (AMT) is the process of converting raw music audio into sheet music that musicians can read [3]. Historically, these tasks have faced challenges such as significant audio noise, long training times, and lack of free-use data due to copyright restrictions. However, recent developments in deep learning have brought new promising approaches to building low-distortion stems and generating sheet music from audio signals [4]. Using spectrogram masking, deep neural networks, and the MuseScore API, we attempt to create an end-to-end pipeline that allows for an initial music audio mixture (e.g...wav file) to be separated into instrument stems, converted into MIDI files, and transcribed into sheet music for each component instrument.