x-vectors meet emotions: A study on dependencies between emotion and speaker recognition
This work addresses the interaction between speaker and emotion recognition for the speech processing community, but it is incremental as it builds on existing methods like x-vectors.
The study investigated the dependencies between speaker and emotion recognition, showing that speaker recognition features can be reused for emotion recognition with transfer learning, achieving absolute improvements of 30.40%, 7.99%, and 8.61% on three datasets after fine-tuning, and found that speaker verification performance degrades with angry utterances.
In this work, we explore the dependencies between speaker recognition and emotion recognition. We first show that knowledge learned for speaker recognition can be reused for emotion recognition through transfer learning. Then, we show the effect of emotion on speaker recognition. For emotion recognition, we show that using a simple linear model is enough to obtain good performance on the features extracted from pre-trained models such as the x-vector model. Then, we improve emotion recognition performance by fine-tuning for emotion classification. We evaluated our experiments on three different types of datasets: IEMOCAP, MSP-Podcast, and Crema-D. By fine-tuning, we obtained 30.40%, 7.99%, and 8.61% absolute improvement on IEMOCAP, MSP-Podcast, and Crema-D respectively over baseline model with no pre-training. Finally, we present results on the effect of emotion on speaker verification. We observed that speaker verification performance is prone to changes in test speaker emotions. We found that trials with angry utterances performed worst in all three datasets. We hope our analysis will initiate a new line of research in the speaker recognition community.