Probing the phonetic and phonological knowledge of tones in Mandarin TTS models
This addresses the problem of linguistic accuracy in TTS models for Mandarin speakers and developers, but it is incremental as it builds on existing methods with minor enhancements.
The study investigated how Mandarin TTS models handle lexical tones, finding that models like Tacotron 2 with BERT embeddings captured tonal coarticulation well but had low accuracy (overall low) in applying Tone-3 sandhi rules to novel sentences, though BERT improved naturalness and generalization.
This study probes the phonetic and phonological knowledge of lexical tones in TTS models through two experiments. Controlled stimuli for testing tonal coarticulation and tone sandhi in Mandarin were fed into Tacotron 2 and WaveGlow to generate speech samples, which were subject to acoustic analysis and human evaluation. Results show that both baseline Tacotron 2 and Tacotron 2 with BERT embeddings capture the surface tonal coarticulation patterns well but fail to consistently apply the Tone-3 sandhi rule to novel sentences. Incorporating pre-trained BERT embeddings into Tacotron 2 improves the naturalness and prosody performance, and yields better generalization of Tone-3 sandhi rules to novel complex sentences, although the overall accuracy for Tone-3 sandhi was still low. Given that TTS models do capture some linguistic phenomena, it is argued that they can be used to generate and validate certain linguistic hypotheses. On the other hand, it is also suggested that linguistically informed stimuli should be included in the training and the evaluation of TTS models.