Deep Remix: Remixing Musical Mixtures Using a Convolutional Deep Neural Network
This work addresses remixing needs for audio engineers or musicians by enabling subtle vocal adjustments in existing mixes, though it is incremental as it builds on prior source separation methods.
The paper tackled the problem of audio source separation for remixing by using a convolutional DNN to estimate ideal binary masks for adjusting vocal balance in musical mixtures, demonstrating that small vocal gain changes can be applied with minimal distortion.
Audio source separation is a difficult machine learning problem and performance is measured by comparing extracted signals with the component source signals. However, if separation is motivated by the ultimate goal of re-mixing then complete separation is not necessary and hence separation difficulty and separation quality are dependent on the nature of the re-mix. Here, we use a convolutional deep neural network (DNN), trained to estimate 'ideal' binary masks for separating voice from music, to perform re-mixing of the vocal balance by operating directly on the individual magnitude components of the musical mixture spectrogram. Our results demonstrate that small changes in vocal gain may be applied with very little distortion to the ultimate re-mix. Our method may be useful for re-mixing existing mixes.