MHTTS: Fast multi-head text-to-speech for spontaneous speech with imperfect transcription
This addresses the challenge of efficiently utilizing spontaneous speech data for TTS, which is incremental as it builds on existing neural TTS methods.
The paper tackles the problem of using massive spontaneous speech with imperfect transcription for text-to-speech synthesis, proposing MHTTS, which synthesizes better quality multi-speaker voice with faster inference speed by transferring text information from high-quality to imperfect data.
Neural network based end-to-end Text-to-Speech (TTS) has greatly improved the quality of synthesized speech. While how to use massive spontaneous speech without transcription efficiently still remains an open problem. In this paper, we propose MHTTS, a fast multi-speaker TTS system that is robust to transcription errors and speaking style speech data. Specifically, we introduce a multi-head model and transfer text information from high-quality corpus with manual transcription to spontaneous speech with imperfectly recognized transcription by jointly training them. MHTTS has three advantages: 1) Our system synthesizes better quality multi-speaker voice with faster inference speed. 2) Our system is capable of transferring correct text information to data with imperfect transcription, simulated using corruption, or provided by an Automatic Speech Recogniser (ASR). 3) Our system can utilize massive real spontaneous speech with imperfect transcription and synthesize expressive voice.