Limited Data Emotional Voice Conversion Leveraging Text-to-Speech: Two-stage Sequence-to-Sequence Training
This work addresses the challenge of generating expressive emotional speech for applications like virtual assistants or entertainment, but it is incremental as it builds on existing sequence-to-sequence and TTS methods.
The paper tackles the problem of emotional voice conversion with limited emotional speech data by proposing a two-stage sequence-to-sequence training strategy that leverages text-to-speech for style initialization and emotion training, achieving significant improvements over state-of-the-art baselines in objective and subjective evaluations.
Emotional voice conversion (EVC) aims to change the emotional state of an utterance while preserving the linguistic content and speaker identity. In this paper, we propose a novel 2-stage training strategy for sequence-to-sequence emotional voice conversion with a limited amount of emotional speech data. We note that the proposed EVC framework leverages text-to-speech (TTS) as they share a common goal that is to generate high-quality expressive voice. In stage 1, we perform style initialization with a multi-speaker TTS corpus, to disentangle speaking style and linguistic content. In stage 2, we perform emotion training with a limited amount of emotional speech data, to learn how to disentangle emotional style and linguistic information from the speech. The proposed framework can perform both spectrum and prosody conversion and achieves significant improvement over the state-of-the-art baselines in both objective and subjective evaluation.