EmoCat: Language-agnostic Emotional Voice Conversion
This work addresses the challenge of generating emotional speech data for downstream tasks in low-resource languages, though it is incremental as it builds on an existing voice conversion system.
The paper tackles the problem of emotional voice conversion with limited language-specific data by proposing EmoCat, a language-agnostic model that achieves high-quality emotion conversion in German using less than 45 minutes of German emotional recordings by leveraging English data, with audio quality on par with recordings for five out of six tested emotion intensities.
Emotional voice conversion models adapt the emotion in speech without changing the speaker identity or linguistic content. They are less data hungry than text-to-speech models and allow to generate large amounts of emotional data for downstream tasks. In this work we propose EmoCat, a language-agnostic emotional voice conversion model. It achieves high-quality emotion conversion in German with less than 45 minutes of German emotional recordings by exploiting large amounts of emotional data in US English. EmoCat is an encoder-decoder model based on CopyCat, a voice conversion system which transfers prosody. We use adversarial training to remove emotion leakage from the encoder to the decoder. The adversarial training is improved by a novel contribution to gradient reversal to truly reverse gradients. This allows to remove only the leaking information and to converge to better optima with higher conversion performance. Evaluations show that Emocat can convert to different emotions but misses on emotion intensity compared to the recordings, especially for very expressive emotions. EmoCat is able to achieve audio quality on par with the recordings for five out of six tested emotion intensities.