LexMatcher: Dictionary-centric Data Collection for LLM-based Machine Translation
This work addresses a data-centric bottleneck for researchers and practitioners in machine translation, offering an incremental improvement over existing fine-tuning approaches.
The paper tackles the problem of data collection for instruction fine-tuning in machine language translation by introducing LexMatcher, a dictionary-centric method for data curation, which outperforms baselines on WMT2022 test sets and improves word sense disambiguation and specialized terminology translation.
The fine-tuning of open-source large language models (LLMs) for machine translation has recently received considerable attention, marking a shift towards data-centric research from traditional neural machine translation. However, the area of data collection for instruction fine-tuning in machine translation remains relatively underexplored. In this paper, we present LexMatcher, a simple yet effective method for data curation, the design of which is driven by the coverage of senses found in bilingual dictionaries. The construction process comprises data retrieval from an existing corpus and data augmentation that supplements the infrequent senses of polysemous words. Utilizing LLaMA2 as our base model, our approach outperforms the established baselines on the WMT2022 test sets and also exhibits remarkable performance in tasks related to word sense disambiguation and specialized terminology translation. These results underscore the effectiveness of LexMatcher in enhancing LLM-based machine translation. The code, data, and models are available at https://github.com/ARIES-LM/Lexmatcher-MT.git.