Multi-channel Speech Separation Using Spatially Selective Deep Non-linear Filters
This work addresses the problem of improving speech separation in multi-channel audio for applications like hearing aids or communication systems, but it is incremental as it builds on existing deep learning methods with a novel spatial filtering approach.
The paper tackled multi-channel speech separation with multiple speakers by proposing a spatially selective deep non-linear filter that uses target direction to enhance spatial processing, showing clear advantage over direct separation when more than two speakers are present and better generalization to unseen noise and similar-angle scenarios.
In a multi-channel separation task with multiple speakers, we aim to recover all individual speech signals from the mixture. In contrast to single-channel approaches, which rely on the different spectro-temporal characteristics of the speech signals, multi-channel approaches should additionally utilize the different spatial locations of the sources for a more powerful separation especially when the number of sources increases. To enhance the spatial processing in a multi-channel source separation scenario, in this work, we propose a deep neural network (DNN) based spatially selective filter (SSF) that can be spatially steered to extract the speaker of interest by initializing a recurrent neural network layer with the target direction. We compare the proposed SSF with a common end-to-end direct separation (DS) approach trained using utterance-wise permutation invariant training (PIT), which only implicitly learns to perform spatial filtering. We show that the SSF has a clear advantage over a DS approach with the same underlying network architecture when there are more than two speakers in the mixture, which can be attributed to a better use of the spatial information. Furthermore, we find that the SSF generalizes much better to additional noise sources that were not seen during training and to scenarios with speakers positioned at a similar angle.