The Effects of Input Type and Pronunciation Dictionary Usage in Transfer Learning for Low-Resource Text-to-Speech
This work addresses the challenge of developing TTS systems for low-resource languages, which often lack pronunciation dictionaries, offering incremental improvements through input type comparisons and new dictionary-free methods.
The study tackled the problem of improving text-to-speech for low-resource languages by comparing phone labels and articulatory features as inputs, finding that articulatory features outperformed phone labels in intelligibility and naturalness for West Frisian, and proposed novel approaches like using a multilingual G2P model that performed on par with ground-truth dictionaries.
We compare phone labels and articulatory features as input for cross-lingual transfer learning in text-to-speech (TTS) for low-resource languages (LRLs). Experiments with FastSpeech 2 and the LRL West Frisian show that using articulatory features outperformed using phone labels in both intelligibility and naturalness. For LRLs without pronunciation dictionaries, we propose two novel approaches: a) using a massively multilingual model to convert grapheme-to-phone (G2P) in both training and synthesizing, and b) using a universal phone recognizer to create a makeshift dictionary. Results show that the G2P approach performs largely on par with using a ground-truth dictionary and the phone recognition approach, while performing generally worse, remains a viable option for LRLs less suitable for the G2P approach. Within each approach, using articulatory features as input outperforms using phone labels.