A Study of Modeling Rising Intonation in Cantonese Neural Speech Synthesis
This addresses a domain-specific issue for Cantonese speech synthesis users by improving intonation modeling, but it is incremental as it builds on existing methods with a classifier.
The paper tackled the problem of synthesizing rising intonation for declarative questions in Cantonese neural TTS, which vanilla systems fail to do due to semantic loss, and found that a BERT-based classifier with separate training achieved the best generalization and feasibility.
In human speech, the attitude of a speaker cannot be fully expressed only by the textual content. It has to come along with the intonation. Declarative questions are commonly used in daily Cantonese conversations, and they are usually uttered with rising intonation. Vanilla neural text-to-speech (TTS) systems are not capable of synthesizing rising intonation for these sentences due to the loss of semantic information. Though it has become more common to complement the systems with extra language models, their performance in modeling rising intonation is not well studied. In this paper, we propose to complement the Cantonese TTS model with a BERT-based statement/question classifier. We design different training strategies and compare their performance. We conduct our experiments on a Cantonese corpus named CanTTS. Empirical results show that the separate training approach obtains the best generalization performance and feasibility.