DirectQE: Direct Pretraining for Machine Translation Quality Estimation
This work addresses a specific bottleneck in QE for machine translation, offering an incremental improvement by bridging data and objective gaps to enhance performance.
The paper tackles the problem of gaps between predictor and estimator in Machine Translation Quality Estimation (QE) by proposing DirectQE, a framework that uses direct pretraining with pseudo data and novel objectives, resulting in outperforming existing methods on benchmarks without relying on pretrained models like BERT.
Machine Translation Quality Estimation (QE) is a task of predicting the quality of machine translations without relying on any reference. Recently, the predictor-estimator framework trains the predictor as a feature extractor, which leverages the extra parallel corpora without QE labels, achieving promising QE performance. However, we argue that there are gaps between the predictor and the estimator in both data quality and training objectives, which preclude QE models from benefiting from a large number of parallel corpora more directly. We propose a novel framework called DirectQE that provides a direct pretraining for QE tasks. In DirectQE, a generator is trained to produce pseudo data that is closer to the real QE data, and a detector is pretrained on these data with novel objectives that are akin to the QE task. Experiments on widely used benchmarks show that DirectQE outperforms existing methods, without using any pretraining models such as BERT. We also give extensive analyses showing how fixing the two gaps contributes to our improvements.