Learning Repeatable Speech Embeddings Using An Intra-class Correlation Regularizer
This work addresses the need for more reliable and repeatable embeddings in speech processing tasks, though it is incremental as it builds on existing contrastive loss methods.
The authors tackled the problem of learning speech embeddings that are sensitive only to the label of interest and invariant to confounding factors by proposing an intra-class correlation regularizer. They demonstrated improved repeatability and downstream task performance on speaker verification, voice style conversion, and dysphonia detection.
A good supervised embedding for a specific machine learning task is only sensitive to changes in the label of interest and is invariant to other confounding factors. We leverage the concept of repeatability from measurement theory to describe this property and propose to use the intra-class correlation coefficient (ICC) to evaluate the repeatability of embeddings. We then propose a novel regularizer, the ICC regularizer, as a complementary component for contrastive losses to guide deep neural networks to produce embeddings with higher repeatability. We use simulated data to explain why the ICC regularizer works better on minimizing the intra-class variance than the contrastive loss alone. We implement the ICC regularizer and apply it to three speech tasks: speaker verification, voice style conversion, and a clinical application for detecting dysphonic voice. The experimental results demonstrate that adding an ICC regularizer can improve the repeatability of learned embeddings compared to only using the contrastive loss; further, these embeddings lead to improved performance in these downstream tasks.