Within-sample variability-invariant loss for robust speaker recognition under noisy environments
This work addresses the challenge of robust speaker verification for applications in noisy real-world settings, representing an incremental improvement over existing methods.
The paper tackles the problem of speaker recognition performance degradation in noisy environments by training a speaker embedding network to learn clean embeddings from noisy utterances using an auxiliary within-sample variability-invariant loss and a data preparation strategy for generating noisy copies on-the-fly. Experiments on VoxCeleb1 show improved performance in both clean and noisy conditions.
Despite the significant improvements in speaker recognition enabled by deep neural networks, unsatisfactory performance persists under noisy environments. In this paper, we train the speaker embedding network to learn the "clean" embedding of the noisy utterance. Specifically, the network is trained with the original speaker identification loss with an auxiliary within-sample variability-invariant loss. This auxiliary variability-invariant loss is used to learn the same embedding among the clean utterance and its noisy copies and prevents the network from encoding the undesired noises or variabilities into the speaker representation. Furthermore, we investigate the data preparation strategy for generating clean and noisy utterance pairs on-the-fly. The strategy generates different noisy copies for the same clean utterance at each training step, helping the speaker embedding network generalize better under noisy environments. Experiments on VoxCeleb1 indicate that the proposed training framework improves the performance of the speaker verification system in both clean and noisy conditions.