Accurate Emotion Strength Assessment for Seen and Unseen Speech Based on Data-Driven Deep Learning
This work addresses the need for robust emotion strength assessment in applications like emotional text-to-speech and voice conversion, though it is incremental as it builds on multi-task learning with existing data fusion techniques.
The paper tackled the problem of emotion strength assessment for speech, which generalizes poorly to unseen domains with existing SVM-based methods, and proposed StrengthNet, a data-driven deep learning model that achieved high correlation with ground truth scores for both seen and unseen speech.
Emotion classification of speech and assessment of the emotion strength are required in applications such as emotional text-to-speech and voice conversion. The emotion attribute ranking function based on Support Vector Machine (SVM) was proposed to predict emotion strength for emotional speech corpus. However, the trained ranking function doesn't generalize to new domains, which limits the scope of applications, especially for out-of-domain or unseen speech. In this paper, we propose a data-driven deep learning model, i.e. StrengthNet, to improve the generalization of emotion strength assessment for seen and unseen speech. This is achieved by the fusion of emotional data from various domains. We follow a multi-task learning network architecture that includes an acoustic encoder, a strength predictor, and an auxiliary emotion predictor. Experiments show that the predicted emotion strength of the proposed StrengthNet is highly correlated with ground truth scores for both seen and unseen speech. We release the source codes at: https://github.com/ttslr/StrengthNet.