A Unified Neural Architecture for Instrumental Audio Tasks
This work addresses the need for flexible and efficient methods in MIR, though it appears incremental as it adapts existing techniques like cGANs and WaveNet to this domain.
The authors tackled the problem of specialized methods for different Music Information Retrieval (MIR) tasks by proposing a unified neural architecture that performs pitch-tracking, source-separation, super-resolution, and synthesis, resulting in the first application of GANs to guided instrument synthesis.
Within Music Information Retrieval (MIR), prominent tasks -- including pitch-tracking, source-separation, super-resolution, and synthesis -- typically call for specialised methods, despite their similarities. Conditional Generative Adversarial Networks (cGANs) have been shown to be highly versatile in learning general image-to-image translations, but have not yet been adapted across MIR. In this work, we present an end-to-end supervisable architecture to perform all aforementioned audio tasks, consisting of a WaveNet synthesiser conditioned on the output of a jointly-trained cGAN spectrogram translator. In doing so, we demonstrate the potential of such flexible techniques to unify MIR tasks, promote efficient transfer learning, and converge research to the improvement of powerful, general methods. Finally, to the best of our knowledge, we present the first application of GANs to guided instrument synthesis.