Bespoke Neural Networks for Score-Informed Source Separation
This addresses the challenge of source separation for music production and analysis, but it is incremental as it builds on existing neural network approaches with a novel training method.
The paper tackles the problem of separating arbitrary musical instruments from audio mixtures using only unaligned MIDI transcriptions, achieving results in a fraction of the time of existing methods.
In this paper, we introduce a simple method that can separate arbitrary musical instruments from an audio mixture. Given an unaligned MIDI transcription for a target instrument from an input mixture, we synthesize new mixtures from the midi transcription that sound similar to the mixture to be separated. This lets us create a labeled training set to train a network on the specific bespoke task. When this model applied to the original mixture, we demonstrate that this method can: 1) successfully separate out the desired instrument with access to only unaligned MIDI, 2) separate arbitrary instruments, and 3) get results in a fraction of the time of existing methods. We encourage readers to listen to the demos posted here: https://git.io/JUu5q.