Generalization Challenges for Neural Architectures in Audio Source Separation
This addresses the problem of robust audio source separation for real-world applications, though it is incremental as it builds on existing neural network approaches.
The paper tackles audio source separation by comparing convolutional neural networks (CNNs) to recurrent neural networks (RNNs), showing that CNNs achieve state-of-the-art results with an order of magnitude fewer parameters and demonstrate superior generalization to new environments like the RealTalkLibri dataset.
Recent work has shown that recurrent neural networks can be trained to separate individual speakers in a sound mixture with high fidelity. Here we explore convolutional neural network models as an alternative and show that they achieve state-of-the-art results with an order of magnitude fewer parameters. We also characterize and compare the robustness and ability of these different approaches to generalize under three different test conditions: longer time sequences, the addition of intermittent noise, and different datasets not seen during training. For the last condition, we create a new dataset, RealTalkLibri, to test source separation in real-world environments. We show that the acoustics of the environment have significant impact on the structure of the waveform and the overall performance of neural network models, with the convolutional model showing superior ability to generalize to new environments. The code for our study is available at https://github.com/ShariqM/source_separation.