Multi-Decoder DPRNN: High Accuracy Source Counting and Separation
This work addresses the problem of speech separation with a variable number of speakers, which is a common challenge in real-world audio processing for various applications.
This paper proposes an end-to-end trainable Multi-Decoder DPRNN for single-channel speech separation with an unknown number of speakers. The model achieves state-of-the-art performance in speaker counting and maintains competitive signal reconstruction quality on WSJ0-mix datasets with up to five speakers.
We propose an end-to-end trainable approach to single-channel speech separation with unknown number of speakers. Our approach extends the MulCat source separation backbone with additional output heads: a count-head to infer the number of speakers, and decoder-heads for reconstructing the original signals. Beyond the model, we also propose a metric on how to evaluate source separation with variable number of speakers. Specifically, we cleared up the issue on how to evaluate the quality when the ground-truth hasmore or less speakers than the ones predicted by the model. We evaluate our approach on the WSJ0-mix datasets, with mixtures up to five speakers. We demonstrate that our approach outperforms state-of-the-art in counting the number of speakers and remains competitive in quality of reconstructed signals.