Benchmarking Large Language Models for Grapheme-to-Phoneme Conversion: A Japanese Case Study
For Japanese text-to-speech, this work shows that LLMs can significantly improve G2P accuracy over traditional methods.
The paper benchmarks over 30 LLMs on Japanese G2P conversion, finding that the best LLMs achieve a kana character error rate below 0.52%, outperforming the best conventional tool (1.03%).
Grapheme-to-phoneme (G2P) conversion is essential for controllable and robust text-to-speech, and large language models (LLMs), with broad linguistic knowledge, offer a promising approach. We benchmarked over 30 LLMs on Japanese G2P, comparing them with conventional morphological analyzers on 3000 manually annotated sentences. We evaluated two prompting strategies: a parse mode, where the LLM performs morphological analysis followed by rule-based kana conversion, and a direct mode, where the LLM directly predicts kana readings. The results show that model size, version, and Japanese-specialized training are key factors, with the best LLMs achieving kana character error rate below 0.52\% vs. the best conventional tool (1.03\%). Parse mode outperforms direct mode for most models, as rule-based post-processing relieves the LLM of handling complex pronunciation rules. We also show that feeding LLM-predicted kana into a kana-input TTS yields better pronunciation than end-to-end TTS.