Mixup-breakdown: a consistency training method for improving generalization of speech separation models
This addresses generalization issues in speech separation for applications like audio processing, though it is incremental as it builds on existing consistency training methods.
The paper tackles the poor generalization of deep-learning speech separation models in mismatch conditions by proposing Mixup-Breakdown training, a consistency-based semi-supervised learning method, which achieves up to 13.77% relative SI-SNRi improvement over strong baselines.
Deep-learning based speech separation models confront poor generalization problem that even the state-of-the-art models could abruptly fail when evaluating them in mismatch conditions. To address this problem, we propose an easy-to-implement yet effective consistency based semi-supervised learning (SSL) approach, namely Mixup-Breakdown training (MBT). It learns a teacher model to "breakdown" unlabeled inputs, and the estimated separations are interpolated to produce more useful pseudo "mixup" input-output pairs, on which the consistency regularization could apply for learning a student model. In our experiment, we evaluate MBT under various conditions with ascending degrees of mismatch, including unseen interfering speech, noise, and music, and compare MBT's generalization capability against state-of-the-art supervised learning and SSL approaches. The result indicates that MBT significantly outperforms several strong baselines with up to 13.77% relative SI-SNRi improvement. Moreover, MBT only adds negligible computational overhead to standard training schemes.