JGU Mainz's Submission to the WMT25 Shared Task on LLMs with Limited Resources for Slavic Languages: MT and QA
This work addresses the problem of limited-resource NLP for Slavic languages, which is incremental as it applies existing fine-tuning techniques to new language-specific tasks.
The paper tackled machine translation and question answering for Slavic languages with limited resources, focusing on Ukrainian, Upper Sorbian, and Lower Sorbian, by fine-tuning a Qwen2.5-3B-Instruct model with parameter-efficient methods and additional data, resulting in models that outperformed the baseline on both tasks.
This paper presents the JGU Mainz submission to the WMT25 Shared Task on LLMs with Limited Resources for Slavic Languages: Machine Translation and Question Answering, focusing on Ukrainian, Upper Sorbian, and Lower Sorbian. For each language, we jointly fine-tune a Qwen2.5-3B-Instruct model for both tasks with parameter-efficient finetuning. Our pipeline integrates additional translation and multiple-choice question answering (QA) data. For Ukrainian QA, we further use retrieval-augmented generation. We also apply ensembling for QA in Upper and Lower Sorbian. Experiments show that our models outperform the baseline on both tasks.