Effect of data reduction on sequence-to-sequence neural TTS
This addresses data efficiency in speech synthesis for applications requiring high-quality TTS with limited speaker-specific data, representing an incremental improvement over existing methods.
The paper tackles the problem of data scarcity in neural text-to-speech by showing that blending data from multiple speakers can compensate for limited target speaker data, achieving better naturalness with 5k utterances from 7 speakers than speaker-dependent models with 15k utterances, and outperforming state-of-the-art systems with significantly less data.
Recent speech synthesis systems based on sampling from autoregressive neural networks models can generate speech almost undistinguishable from human recordings. However, these models require large amounts of data. This paper shows that the lack of data from one speaker can be compensated with data from other speakers. The naturalness of Tacotron2-like models trained on a blend of 5k utterances from 7 speakers is better than that of speaker dependent models trained on 15k utterances, but in terms of stability multi-speaker models are always more stable. We also demonstrate that models mixing only 1250 utterances from a target speaker with 5k utterances from another 6 speakers can produce significantly better quality than state-of-the-art DNN-guided unit selection systems trained on more than 10 times the data from the target speaker.