Prompting with Phonemes: Enhancing LLMs' Multilinguality for Non-Latin Script Languages
This addresses the performance gap in multilingual LLMs for non-Latin script languages, which is an incremental improvement for NLP applications in diverse linguistic contexts.
The paper tackles the underperformance of multilingual LLMs on non-Latin script languages by using phonemic transcriptions to create script-invariant representations, resulting in performance improvements of up to 12.6% for Latin script languages and up to 15.1% for non-Latin script languages compared to randomized retrieval.
Although multilingual LLMs have achieved remarkable performance across benchmarks, we find they continue to underperform on non-Latin script languages across contemporary LLM families. This discrepancy arises from the fact that LLMs are pretrained with orthographic scripts, which are dominated by Latin characters that obscure their shared phonology with non-Latin scripts. We propose leveraging phonemic transcriptions as complementary signals to induce script-invariant representations. Our study demonstrates that integrating phonemic signals improves performance across both non-Latin and Latin script languages, with a particularly significant impact on closing the performance gap between the two. Through detailed experiments, we show that phonemic and orthographic scripts retrieve distinct examples for in-context learning (ICL). This motivates our proposed Mixed-ICL retrieval strategy, where further aggregation from both leads to our significant performance improvements for both Latin script languages (up to 12.6%) and non-Latin script languages (up to 15.1%) compared to randomized ICL retrieval.