Stabilizing Label Assignment for Speech Separation by Self-supervised Pre-training
This work addresses a specific issue in speech separation training for researchers and practitioners, offering incremental improvements to existing methods.
The paper tackles the problem of frequent label assignment switching in permutation invariant training for speech separation, which hinders convergence speed and performance, and shows that self-supervised pre-training can achieve very good improvements when a proper approach is chosen.
Speech separation has been well developed, with the very successful permutation invariant training (PIT) approach, although the frequent label assignment switching happening during PIT training remains to be a problem when better convergence speed and achievable performance are desired. In this paper, we propose to perform self-supervised pre-training to stabilize the label assignment in training the speech separation model. Experiments over several types of self-supervised approaches, several typical speech separation models and two different datasets showed that very good improvements are achievable if a proper self-supervised approach is chosen.