Multilingual Prosody Transfer: Comparing Supervised & Transfer Learning
This work addresses building better text-to-speech models for low-resource languages, representing an incremental advance in multilingual speech synthesis.
The study compared supervised fine-tuning and transfer learning for adapting monolingual TTS models to multilingual prosody transfer, finding that transfer learning significantly outperformed with an average MOS higher by 1.53 points, a 37.5% increase in recognition accuracy, and a 7.8-point improvement in MCD.
The field of prosody transfer in speech synthesis systems is rapidly advancing. This research is focused on evaluating learning methods for adapting pre-trained monolingual text-to-speech (TTS) models to multilingual conditions, i.e., Supervised Fine-Tuning (SFT) and Transfer Learning (TL). This comparison utilizes three distinct metrics: Mean Opinion Score (MOS), Recognition Accuracy (RA), and Mel Cepstral Distortion (MCD). Results demonstrate that, in comparison to SFT, TL leads to significantly enhanced performance, with an average MOS higher by 1.53 points, a 37.5% increase in RA, and approximately a 7.8-point improvement in MCD. These findings are instrumental in helping build TTS models for low-resource languages.