Using previous acoustic context to improve Text-to-Speech synthesis
This work addresses the problem of improving the naturalness of synthesized speech for users of Text-to-Speech systems by leveraging previously ignored sequential data characteristics.
The authors propose a method to improve Text-to-Speech synthesis by incorporating acoustic context from previous utterances, which are often discarded in current models. They achieve significant improvements in naturalness over a Tacotron 2 baseline by using an acoustic context encoder and a secondary task for additional supervision.
Many speech synthesis datasets, especially those derived from audiobooks, naturally comprise sequences of utterances. Nevertheless, such data are commonly treated as individual, unordered utterances both when training a model and at inference time. This discards important prosodic phenomena above the utterance level. In this paper, we leverage the sequential nature of the data using an acoustic context encoder that produces an embedding of the previous utterance audio. This is input to the decoder in a Tacotron 2 model. The embedding is also used for a secondary task, providing additional supervision. We compare two secondary tasks: predicting the ordering of utterance pairs, and predicting the embedding of the current utterance audio. Results show that the relation between consecutive utterances is informative: our proposed model significantly improves naturalness over a Tacotron 2 baseline.