EmoPro: A Prompt Selection Strategy for Emotional Expression in LM-based Speech Synthesis
This work addresses the challenge of fine-tuning emotional expression in speech synthesis for applications requiring nuanced emotional control, representing an incremental improvement over existing prompt selection schemes.
The paper tackled the problem of prompt selection for controlling emotional intensity in speech synthesis, proposing a two-stage strategy called EmoPro that selects prompts based on emotional expression strength, speech quality, text-emotion consistency, and model generation performance, resulting in more emotionally expressive and engaging synthesized speech compared to baseline methods.
Recent advancements in speech synthesis models, trained on extensive datasets, have demonstrated remarkable zero-shot capabilities. These models can control content, timbre, and emotion in generated speech based on prompt inputs. Despite these advancements, the choice of prompts significantly impacts the output quality, yet most existing selection schemes do not adequately address the control of emotional intensity. To address this question, this paper proposes a two-stage prompt selection strategy EmoPro, which is specifically designed for emotionally controllable speech synthesis. This strategy focuses on selecting highly expressive and high-quality prompts by evaluating them from four perspectives: emotional expression strength, speech quality, text-emotion consistency, and model generation performance. Experimental results show that prompts selected using the proposed method result in more emotionally expressive and engaging synthesized speech compared to those obtained through baseline. Audio samples and codes will be available at https://whyrrrrun.github.io/EmoPro/.