Score Combination for Improved Parallel Corpus Filtering for Low Resource Conditions
This work addresses the challenge of improving machine translation quality for low-resource languages, though it is incremental as it builds on existing methods and datasets.
The paper tackled the problem of parallel corpus filtering for low-resource languages by combining multiple scoring methods, resulting in a 7% relative improvement in sacreBLEU for Pashto and 5% for Khmer over a baseline in a WMT20 task.
This paper describes our submission to the WMT20 sentence filtering task. We combine scores from (1) a custom LASER built for each source language, (2) a classifier built to distinguish positive and negative pairs by semantic alignment, and (3) the original scores included in the task devkit. For the mBART finetuning setup, provided by the organizers, our method shows 7% and 5% relative improvement over baseline, in sacreBLEU score on the test set for Pashto and Khmer respectively.