Enhancement of Spatial Clustering-Based Time-Frequency Masks using LSTM Neural Networks
This work provides an incremental improvement for speech enhancement systems by combining existing methods to improve performance and generalization across varying microphone configurations.
This paper addresses the challenge of combining the strengths of deep recurrent neural networks (LSTM) for single-channel speech enhancement and spatial clustering for multi-channel generalization. The proposed system enhances spatial clustering-based time-frequency masks using LSTMs, achieving both strong signal modeling and generalizable signal separation.
Recent works have shown that Deep Recurrent Neural Networks using the LSTM architecture can achieve strong single-channel speech enhancement by estimating time-frequency masks. However, these models do not naturally generalize to multi-channel inputs from varying microphone configurations. In contrast, spatial clustering techniques can achieve such generalization but lack a strong signal model. Our work proposes a combination of the two approaches. By using LSTMs to enhance spatial clustering based time-frequency masks, we achieve both the signal modeling performance of multiple single-channel LSTM-DNN speech enhancers and the signal separation performance and generality of multi-channel spatial clustering. We compare our proposed system to several baselines on the CHiME-3 dataset. We evaluate the quality of the audio from each system using SDR from the BSS\_eval toolkit and PESQ. We evaluate the intelligibility of the output of each system using word error rate from a Kaldi automatic speech recognizer.