Task Vector in TTS: Toward Emotionally Expressive Dialectal Speech Synthesis
This work addresses a specific problem in text-to-speech for generating cross-style speech, but it is incremental as it builds on existing TTS advancements.
The paper tackles the challenge of synthesizing emotionally expressive dialectal speech, which is difficult due to a lack of labeled data, by proposing a two-stage method called HE-Vector that achieves superior dialect synthesis and promising zero-shot results for emotional dialect synthesis.
Recent advances in text-to-speech (TTS) have yielded remarkable improvements in naturalness and intelligibility. Building on these achievements, research has increasingly shifted toward enhancing the expressiveness of generated speech, such as dialectal and emotional TTS. However, cross-style synthesis combining both dialect and emotion remains challenging and largely unexplored, mainly due to the scarcity of dialectal data with emotional labels. To address this, we propose Hierarchical Expressive Vector (HE-Vector), a two-stage method for Emotional Dialectal TTS. In the first stage, we construct different task vectors to model dialectal and emotional styles independently, and then enhance single-style synthesis by adjusting their weights, a method we refer to as Expressive Vector (E-Vector). For the second stage, we hierarchically integrate these vectors to achieve controllable emotionally expressive dialect synthesis without requiring jointly labeled data, corresponding to Hierarchical Expressive Vector (HE-Vector). Experimental results demonstrate that HE-Vectors achieve superior performance in dialect synthesis, and promising results in synthesizing emotionally expressive dialectal speech in a zero-shot setting.