Unsupervised word-level prosody tagging for controllable speech synthesis
This addresses the problem of controllable speech synthesis for TTS applications, but it is incremental as it builds on existing prosody modeling research.
The paper tackles the challenge of manually controlling word-level prosody in speech synthesis by proposing an unsupervised tagging method, resulting in a TTS model that achieves better naturalness than FastSpeech2 and enables prosody manipulation.
Although word-level prosody modeling in neural text-to-speech (TTS) has been investigated in recent research for diverse speech synthesis, it is still challenging to control speech synthesis manually without a specific reference. This is largely due to lack of word-level prosody tags. In this work, we propose a novel approach for unsupervised word-level prosody tagging with two stages, where we first group the words into different types with a decision tree according to their phonetic content and then cluster the prosodies using GMM within each type of words separately. This design is based on the assumption that the prosodies of different type of words, such as long or short words, should be tagged with different label sets. Furthermore, a TTS system with the derived word-level prosody tags is trained for controllable speech synthesis. Experiments on LJSpeech show that the TTS model trained with word-level prosody tags not only achieves better naturalness than a typical FastSpeech2 model, but also gains the ability to manipulate word-level prosody.