Mismatch Aware Guidance for Robust Emotion Control in Auto-Regressive TTS Models
This addresses a specific challenge in fine-grained emotional control for TTS systems, representing an incremental improvement.
The paper tackled the problem of style-content mismatch in auto-regressive text-to-speech models, which causes unnatural speech, by proposing an adaptive classifier-free guidance scheme that adjusts to mismatch levels, resulting in improved emotional expressiveness while maintaining audio quality and intelligibility.
While Text-to-Speech (TTS) systems can achieve fine-grained control over emotional expression via natural language prompts, a significant challenge emerges when the desired emotion (style prompt) conflicts with the semantic content of the text. This mismatch often results in unnatural-sounding speech, undermining the goal of achieving fine-grained emotional control. Classifier-Free Guidance (CFG) is a key technique for enhancing prompt alignment; however, its application to auto-regressive (AR) TTS models remains underexplored, which can lead to degraded audio quality. This paper directly addresses the challenge of style-content mismatch in AR TTS models by proposing an adaptive CFG scheme that adjusts to different levels of the detected mismatch, as measured using large language models or natural language inference models. This solution is based on a comprehensive analysis of CFG's impact on emotional expressiveness in state-of-the-art AR TTS models. Our results demonstrate that the proposed adaptive CFG scheme improves the emotional expressiveness of the AR TTS model while maintaining audio quality and intelligibility.