There's no Data Like Better Data: Using QE Metrics for MT Data Filtering
This work addresses data efficiency for NMT practitioners by offering a method to filter out low-quality sentence pairs, though it is incremental as it builds on existing QE techniques.
The paper tackled the problem of improving neural machine translation (NMT) by filtering training data using Quality Estimation (QE) metrics, resulting in enhanced translation quality while reducing training data size by half.
Quality Estimation (QE), the evaluation of machine translation output without the need of explicit references, has seen big improvements in the last years with the use of neural metrics. In this paper we analyze the viability of using QE metrics for filtering out bad quality sentence pairs in the training data of neural machine translation systems~(NMT). While most corpus filtering methods are focused on detecting noisy examples in collections of texts, usually huge amounts of web crawled data, QE models are trained to discriminate more fine-grained quality differences. We show that by selecting the highest quality sentence pairs in the training data, we can improve translation quality while reducing the training size by half. We also provide a detailed analysis of the filtering results, which highlights the differences between both approaches.