End-to-end Text-to-speech for Low-resource Languages by Cross-Lingual Transfer Learning
This addresses the problem of laborious data collection for TTS in low-resource languages, which affects over 95% of languages, but it is incremental as it builds on cross-lingual transfer learning.
The paper tackled building text-to-speech systems for low-resource languages with limited paired data by transferring knowledge from a high-resource language using a learned mapping between linguistic symbols, achieving a relatively good system with only around 15 minutes of paired data.
End-to-end text-to-speech (TTS) has shown great success on large quantities of paired text plus speech data. However, laborious data collection remains difficult for at least 95% of the languages over the world, which hinders the development of TTS in different languages. In this paper, we aim to build TTS systems for such low-resource (target) languages where only very limited paired data are available. We show such TTS can be effectively constructed by transferring knowledge from a high-resource (source) language. Since the model trained on source language cannot be directly applied to target language due to input space mismatch, we propose a method to learn a mapping between source and target linguistic symbols. Benefiting from this learned mapping, pronunciation information can be preserved throughout the transferring procedure. Preliminary experiments show that we only need around 15 minutes of paired data to obtain a relatively good TTS system. Furthermore, analytic studies demonstrated that the automatically discovered mapping correlate well with the phonetic expertise.