Multichannel Loss Function for Supervised Speech Source Separation by Mask-based Beamforming
This work addresses a specific bottleneck in supervised speech source separation for applications like audio processing, but it is incremental as it builds on existing mask-based beamforming methods.
The paper tackled the problem of training deep neural networks for mask-based beamforming in speech source separation by proposing multichannel loss functions that evaluate estimated spatial covariance matrices, leading to improved effectiveness and robustness to microphone configurations in experiments.
In this paper, we propose two mask-based beamforming methods using a deep neural network (DNN) trained by multichannel loss functions. Beamforming technique using time-frequency (TF)-masks estimated by a DNN have been applied to many applications where TF-masks are used for estimating spatial covariance matrices. To train a DNN for mask-based beamforming, loss functions designed for monaural speech enhancement/separation have been employed. Although such a training criterion is simple, it does not directly correspond to the performance of mask-based beamforming. To overcome this problem, we use multichannel loss functions which evaluate the estimated spatial covariance matrices based on the multichannel Itakura--Saito divergence. DNNs trained by the multichannel loss functions can be applied to construct several beamformers. Experimental results confirmed their effectiveness and robustness to microphone configurations.