Alternate Endings: Improving Prosody for Incremental Neural TTS with Predicted Future Text Input
This addresses naturalness issues in incremental TTS for real-time applications, but is incremental as it builds on existing prediction methods.
The paper tackled the problem of prosody degradation in incremental text-to-speech synthesis due to lack of future context, and found that using predicted future text significantly improves prosodic features over no lookahead, with slight gains over random predictions.
The prosody of a spoken word is determined by its surrounding context. In incremental text-to-speech synthesis, where the synthesizer produces an output before it has access to the complete input, the full context is often unknown which can result in a loss of naturalness in the synthesized speech. In this paper, we investigate whether the use of predicted future text can attenuate this loss. We compare several test conditions of next future word: (a) unknown (zero-word), (b) language model predicted, (c) randomly predicted and (d) ground-truth. We measure the prosodic features (pitch, energy and duration) and find that predicted text provides significant improvements over a zero-word lookahead, but only slight gains over random-word lookahead. We confirm these results with a perceptive test.