SelfTTS: cross-speaker style transfer through explicit embedding disentanglement and self-refinement using self-augmentation
This addresses the problem of generating emotionally expressive speech from neutral speakers for TTS applications, representing an incremental improvement with novel method components.
The paper tackled cross-speaker style transfer in text-to-speech by developing SelfTTS, which uses explicit embedding disentanglement and self-refinement, resulting in superior emotional naturalness and robust stability compared to state-of-the-art baselines.
This paper presents SelfTTS, a text-to-speech (TTS) model designed for cross-speaker style transfer that eliminates the need for external pre-trained speaker or emotion encoders. The architecture achieves emotional expressivity in neutral speakers through an explicit disentanglement strategy utilizing Gradient Reversal Layers (GRL) combined with cosine similarity loss to decouple speaker and emotion information. We introduce Multi Positive Contrastive Learning (MPCL) to induce clustered representations of speaker and emotion embeddings based on their respective labels. Furthermore, SelfTTS employs a self-refinement strategy via Self-Augmentation, exploiting the model's voice conversion capabilities to enhance the naturalness of synthesized speech. Experimental results demonstrate that SelfTTS achieves superior emotional naturalness (eMOS) and robust stability in target timbre and emotion compared to state-of-the-art baselines.