Emotional Prosody Control for Speech Generation
This work addresses the need for more natural and customizable emotional expression in speech generation for applications like virtual assistants and entertainment, representing an incremental improvement over existing methods.
The authors tackled the problem of limited emotional variation in machine-generated speech by developing a TTS system that allows users to select emotions from a continuous Arousal-Valence space, enabling fine control and scaling to unseen emotions and speakers without degrading speech quality.
Machine-generated speech is characterized by its limited or unnatural emotional variation. Current text to speech systems generates speech with either a flat emotion, emotion selected from a predefined set, average variation learned from prosody sequences in training data or transferred from a source style. We propose a text to speech(TTS) system, where a user can choose the emotion of generated speech from a continuous and meaningful emotion space (Arousal-Valence space). The proposed TTS system can generate speech from the text in any speaker's style, with fine control of emotion. We show that the system works on emotion unseen during training and can scale to previously unseen speakers given his/her speech sample. Our work expands the horizon of the state-of-the-art FastSpeech2 backbone to a multi-speaker setting and gives it much-coveted continuous (and interpretable) affective control, without any observable degradation in the quality of the synthesized speech.