AttentionStitch: How Attention Solves the Speech Editing Problem
This addresses the speech editing task for natural language processing applications, offering an incremental improvement by enhancing existing TTS models for better integration.
The paper tackled the problem of seamlessly integrating edited speech into synthesized speech by proposing AttentionStitch, a method that leverages a pre-trained TTS model with a double attention block network, and demonstrated superior performance on LJSpeech and VCTK datasets through objective and subjective evaluations with 15 human participants.
The generation of natural and high-quality speech from text is a challenging problem in the field of natural language processing. In addition to speech generation, speech editing is also a crucial task, which requires the seamless and unnoticeable integration of edited speech into synthesized speech. We propose a novel approach to speech editing by leveraging a pre-trained text-to-speech (TTS) model, such as FastSpeech 2, and incorporating a double attention block network on top of it to automatically merge the synthesized mel-spectrogram with the mel-spectrogram of the edited text. We refer to this model as AttentionStitch, as it harnesses attention to stitch audio samples together. We evaluate the proposed AttentionStitch model against state-of-the-art baselines on both single and multi-speaker datasets, namely LJSpeech and VCTK. We demonstrate its superior performance through an objective and a subjective evaluation test involving 15 human participants. AttentionStitch is capable of producing high-quality speech, even for words not seen during training, while operating automatically without the need for human intervention. Moreover, AttentionStitch is fast during both training and inference and is able to generate human-sounding edited speech.