LLaMAX2: Your Translation-Enhanced Model also Performs Well in Reasoning
This work addresses the trade-off between translation and reasoning in LLMs, offering a simpler approach for multilingual enhancement, though it appears incremental as it builds on existing models.
The paper tackles the problem of translation-enhanced LLMs struggling with reasoning tasks by proposing a novel recipe using layer-selective tuning on parallel data, resulting in Qwen3-XPlus models that improve translation performance (e.g., 15+ spBLEU and 40+ xComet in low-resource languages) while maintaining reasoning proficiency.
General Large Language Models (LLMs) excel in reasoning, but those enhanced for translation struggle with reasoning tasks. To address this, we propose a novel translationenhanced recipe that begins with instruct models and applies layer-selective tuning only on parallel data. Following this pipeline, we introduce the Qwen3-XPlus models, which demonstrate significant improvements in translation performance across both high- and lowresource languages, achieving 15+ spBLEU and 40+ xComet in low-resource languages, like Swahili. Interestingly, training only with small parallel datasets, Qwen3-XPlus achieves an average improvement of 1+ points on 7 multilingual tasks while maintaining proficiency comparable to the Qwen3 instruct model in 15 popular reasoning datasets. This work offers a promising approach to multilingual enhancement, significantly reducing complexity and enhancing accessibility for a wider range of languages. The code and model are publicly available.