IKUN for WMT24 General MT Task: LLMs Are here for Multilingual Machine Translation
This work addresses the problem of effective multilingual translation for NLP practitioners, showing incremental improvements in LLM-based systems.
The paper tackled multilingual machine translation by developing two systems, IKUN and IKUN-C, based on LLMs like Llama-3-8b and Mistral-7B-v0.3, which achieved top rankings in the WMT24 competition, with IKUN-C securing 6 first-place and 3 second-place finishes among constrained systems.
This paper introduces two multilingual systems, IKUN and IKUN-C, developed for the general machine translation task in WMT24. IKUN and IKUN-C represent an open system and a constrained system, respectively, built on Llama-3-8b and Mistral-7B-v0.3. Both systems are designed to handle all 11 language directions using a single model. According to automatic evaluation metrics, IKUN-C achieved 6 first-place and 3 second-place finishes among all constrained systems, while IKUN secured 1 first-place and 2 second-place finishes across both open and constrained systems. These encouraging results suggest that large language models (LLMs) are nearing the level of proficiency required for effective multilingual machine translation. The systems are based on a two-stage approach: first, continuous pre-training on monolingual data in 10 languages, followed by fine-tuning on high-quality parallel data for 11 language directions. The primary difference between IKUN and IKUN-C lies in their monolingual pre-training strategy. IKUN-C is pre-trained using constrained monolingual data, whereas IKUN leverages monolingual data from the OSCAR dataset. In the second phase, both systems are fine-tuned on parallel data sourced from NTREX, Flores, and WMT16-23 for all 11 language pairs.