Efficient Integration of Multi-channel Information for Speaker-independent Speech Separation
This work addresses a key challenge in speech separation for applications like hearing aids or voice assistants, but it is incremental as it builds on existing single-channel methods.
The paper tackles the problem of integrating multi-channel signals for speaker-independent speech separation by proposing early-fusion and late-fusion methods based on a time-domain audio separation network, along with a channel-sequential-transfer learning framework. The results show that these methods outperform multi-channel deep clustering, improve performance proportionally to the number of microphones, and the late-fusion method consistently exceeds single-channel performance regardless of speaker angle differences.
Although deep-learning-based methods have markedly improved the performance of speech separation over the past few years, it remains an open question how to integrate multi-channel signals for speech separation. We propose two methods, namely, early-fusion and late-fusion methods, to integrate multi-channel information based on the time-domain audio separation network, which has been proven effective in single-channel speech separation. We also propose channel-sequential-transfer learning, which is a transfer learning framework that applies the parameters trained for a lower-channel network as the initial values of a higher-channel network. For fair comparison, we evaluated our proposed methods using a spatialized version of the wsj0-2mix dataset, which is open-sourced. It was found that our proposed methods can outperform multi-channel deep clustering and improve the performance proportionally to the number of microphones. It was also proven that the performance of the late-fusion method is consistently higher than that of the single-channel method regardless of the angle difference between speakers.