Incremental Text-to-Speech Synthesis Using Pseudo Lookahead with Large Pretrained Language Model
This work addresses the trade-off between latency and speech quality in incremental TTS for users who require real-time speech generation.
This paper proposes an incremental text-to-speech (TTS) method that uses a pseudo lookahead generated by a large pretrained language model (GPT2) to maintain naturalness while synthesizing speech in small linguistic units. The method achieves higher speech quality than approaches without future context and matches the quality of methods that wait for actual future context observation.
This letter presents an incremental text-to-speech (TTS) method that performs synthesis in small linguistic units while maintaining the naturalness of output speech. Incremental TTS is generally subject to a trade-off between latency and synthetic speech quality. It is challenging to produce high-quality speech with a low-latency setup that does not make much use of an unobserved future sentence (hereafter, "lookahead"). To resolve this issue, we propose an incremental TTS method that uses a pseudo lookahead generated with a language model to take the future contextual information into account without increasing latency. Our method can be regarded as imitating a human's incremental reading and uses pretrained GPT2, which accounts for the large-scale linguistic knowledge, for the lookahead generation. Evaluation results show that our method 1) achieves higher speech quality than the method taking only observed information into account and 2) achieves a speech quality equivalent to waiting for the future context observation.