ZET-Speech: Zero-shot adaptive Emotion-controllable Text-to-Speech Synthesis with Diffusion and Style-based Models
This addresses the need for more flexible and generalizable emotional TTS in applications like human-like dialogue agents, representing an incremental improvement by extending capabilities to unseen speakers.
The paper tackles the problem of synthesizing emotional speech for unseen speakers in text-to-speech systems, proposing ZET-Speech which uses domain adversarial learning and guidance on diffusion models to achieve zero-shot adaptation with only a neutral speech segment and emotion label, resulting in natural and emotional speech for both seen and unseen speakers.
Emotional Text-To-Speech (TTS) is an important task in the development of systems (e.g., human-like dialogue agents) that require natural and emotional speech. Existing approaches, however, only aim to produce emotional TTS for seen speakers during training, without consideration of the generalization to unseen speakers. In this paper, we propose ZET-Speech, a zero-shot adaptive emotion-controllable TTS model that allows users to synthesize any speaker's emotional speech using only a short, neutral speech segment and the target emotion label. Specifically, to enable a zero-shot adaptive TTS model to synthesize emotional speech, we propose domain adversarial learning and guidance methods on the diffusion model. Experimental results demonstrate that ZET-Speech successfully synthesizes natural and emotional speech with the desired emotion for both seen and unseen speakers. Samples are at https://ZET-Speech.github.io/ZET-Speech-Demo/.