Investigating Stochastic Methods for Prosody Modeling in Speech Synthesis
This addresses the problem of improving prosody modeling for speech synthesis systems, offering incremental advancements in controllability and naturalness.
The paper tackled the challenge of generating expressive prosody in text-to-speech synthesis by investigating stochastic methods like Normalizing Flows, finding that these methods produce natural prosody on par with human speakers and offer controllability through temperature tuning.
While generative methods have progressed rapidly in recent years, generating expressive prosody for an utterance remains a challenging task in text-to-speech synthesis. This is particularly true for systems that model prosody explicitly through parameters such as pitch, energy, and duration, which is commonly done for the sake of interpretability and controllability. In this work, we investigate the effectiveness of stochastic methods for this task, including Normalizing Flows, Conditional Flow Matching, and Rectified Flows. We compare these methods to a traditional deterministic baseline, as well as to real human realizations. Our extensive subjective and objective evaluations demonstrate that stochastic methods produce natural prosody on par with human speakers by capturing the variability inherent in human speech. Further, they open up additional controllability options by allowing the sampling temperature to be tuned.