Code-Switching In-Context Learning for Cross-Lingual Transfer of Large Language Models
This addresses the translation barrier that limits the inclusiveness of LLM-based applications for non-English speakers, representing a novel method for a known bottleneck rather than a foundational breakthrough.
The paper tackles the problem of large language models' performance degradation in non-English languages due to their reliance on implicit translation to English, by introducing code-switching in-context learning (CSICL) which progressively transitions demonstrations from target languages to English. The method consistently outperforms baselines, achieving gains of 3.1%p in target languages and 1.9%p in unseen languages, with even larger improvements of 14.7% and 5.3% in low-resource settings.
While large language models (LLMs) exhibit strong multilingual abilities, their reliance on English as latent representations creates a translation barrier, where reasoning implicitly depends on internal translation into English. When this process fails, performance in non-English languages deteriorates sharply, limiting the inclusiveness of LLM-based applications. Existing cross-lingual in-context learning (X-ICL) methods primarily leverage monolingual demonstrations, often failing to mitigate this barrier and instead reinforcing it. In this work, we introduce code-switching in-context learning (CSICL), a simple yet effective prompting strategy that progressively transitions from a target language to English within demonstrations and instruction to facilitate their latent reasoning in English. By explicitly scaffolding the reasoning process through controlled code-switching, CSICL acts as an implicit linguistic bridge that enhances cross-lingual alignment and reduces reliance on the translation barrier. We conduct extensive experiments across 4 LLMs, 6 datasets, and 10 languages, spanning both knowledge-intensive and reasoning-oriented domains. Our results demonstrate that CSICL consistently outperforms X-ICL baselines, achieving gains of 3.1%p and 1.9%p in both target and unseen languages, respectively. The improvement is even more pronounced in low-resource settings, with gains of 14.7% in target and 5.3% in unseen languages. These findings establish code-switching as a principled and robust approach for overcoming the translation barrier during inference, moving LLMs toward more equitable and effective multilingual systems.