Speaker and Direction Inferred Dual-channel Speech Separation
This work provides a solution for robust speech separation in dynamic, multi-speaker environments, which is a significant problem for speech processing applications.
This paper addresses the challenge of speech separation in real-world scenarios with an uncertain number of speakers by proposing SDNet, a network that sequentially infers speaker and direction characteristics to attend to individual speech sources. The method achieves significant SDR improvements of 25.31 dB on WSJ0-2mix, 17.26 dB on WSJ0-3mix, and 21.56 dB on WSJ0-2&3mix under anechoic settings.
Most speech separation methods, trying to separate all channel sources simultaneously, are still far from having enough general- ization capabilities for real scenarios where the number of input sounds is usually uncertain and even dynamic. In this work, we employ ideas from auditory attention with two ears and propose a speaker and direction inferred speech separation network (dubbed SDNet) to solve the cocktail party problem. Specifically, our SDNet first parses out the respective perceptual representations with their speaker and direction characteristics from the mixture of the scene in a sequential manner. Then, the perceptual representations are utilized to attend to each corresponding speech. Our model gener- ates more precise perceptual representations with the help of spatial features and successfully deals with the problem of the unknown number of sources and the selection of outputs. The experiments on standard fully-overlapped speech separation benchmarks, WSJ0- 2mix, WSJ0-3mix, and WSJ0-2&3mix, show the effectiveness, and our method achieves SDR improvements of 25.31 dB, 17.26 dB, and 21.56 dB under anechoic settings. Our codes will be released at https://github.com/aispeech-lab/SDNet.