A Training and Inference Strategy Using Noisy and Enhanced Speech as Target for Speech Enhancement without Clean Speech
This work addresses a practical challenge in speech enhancement for applications where clean speech data is unavailable, though it is incremental as it improves upon existing noisy-target training methods.
The paper tackles the problem of training speech enhancement systems without access to clean speech by proposing a strategy that uses noisy and enhanced speech as targets, building on noisy-target training. Experimental results show that the method outperforms baselines, particularly with a teacher/student inference approach.
The lack of clean speech is a practical challenge to the development of speech enhancement systems, which means that there is an inevitable mismatch between their training criterion and evaluation metric. In response to this unfavorable situation, we propose a training and inference strategy that additionally uses enhanced speech as a target by improving the previously proposed noisy-target training (NyTT). Because homogeneity between in-domain noise and extraneous noise is the key to the effectiveness of NyTT, we train various student models by remixing 1) the teacher model's estimated speech and noise for enhanced-target training or 2) raw noisy speech and the teacher model's estimated noise for noisy-target training. Experimental results show that our proposed method outperforms several baselines, especially with the teacher/student inference, where predicted clean speech is derived successively through the teacher and final student models.