Cross-speaker style transfer for text-to-speech using data augmentation
This addresses the problem of generating expressive speech for TTS systems while preserving speaker identity, which is incremental as it builds on existing voice conversion and data augmentation techniques.
The paper tackled cross-speaker style transfer for text-to-speech by using data augmentation via voice conversion to generate expressive speech while retaining the target speaker's identity, scaling to 14 speakers across 7 languages with consistent improvements in style similarity.
We address the problem of cross-speaker style transfer for text-to-speech (TTS) using data augmentation via voice conversion. We assume to have a corpus of neutral non-expressive data from a target speaker and supporting conversational expressive data from different speakers. Our goal is to build a TTS system that is expressive, while retaining the target speaker's identity. The proposed approach relies on voice conversion to first generate high-quality data from the set of supporting expressive speakers. The voice converted data is then pooled with natural data from the target speaker and used to train a single-speaker multi-style TTS system. We provide evidence that this approach is efficient, flexible, and scalable. The method is evaluated using one or more supporting speakers, as well as a variable amount of supporting data. We further provide evidence that this approach allows some controllability of speaking style, when using multiple supporting speakers. We conclude by scaling our proposed technology to a set of 14 speakers across 7 languages. Results indicate that our technology consistently improves synthetic samples in terms of style similarity, while retaining the target speaker's identity.