Can you map it to English? The Role of Cross-Lingual Alignment in Multilingual Performance of LLMs
This work addresses the problem of understanding cross-lingual generalization mechanisms in LLMs for researchers and practitioners in multilingual AI, but it is incremental as it builds on existing alignment concepts.
The study investigated how cross-lingual alignment of representations correlates with multilingual LLM performance on tasks like natural language understanding and translation, finding that alignment metrics strongly correlate with accuracy at the language level but not at the sample level, indicating it is necessary but insufficient for success.
Large language models (LLMs) pre-trained predominantly on English text exhibit surprising multilingual capabilities, yet the mechanisms driving cross-lingual generalization remain poorly understood. This work investigates how the alignment of representations for text written in different languages correlates with LLM performance on natural language understanding tasks and translation tasks, both at the language and the instance level. For this purpose, we introduce cross-lingual alignment metrics such as the Discriminative Alignment Index (DALI) to quantify the alignment at an instance level for discriminative tasks. Through experiments on three natural language understanding tasks (Belebele, XStoryCloze, XCOPA), and machine translation, we find that while cross-lingual alignment metrics strongly correlate with task accuracy at the language level, the sample-level alignment often fails to distinguish correct from incorrect predictions, exposing alignment as a necessary but insufficient condition for success.