Unsupervised Translation Quality Estimation Exploiting Synthetic Data and Pre-trained Multilingual Encoder
This work addresses the data scarcity issue in translation quality estimation for various language directions, though it is incremental as it builds on existing components proven in supervised settings.
The paper tackled the problem of unsupervised translation quality estimation by investigating synthetic data and pre-trained multilingual encoders, achieving results that outperform other unsupervised methods on high- and low-resource translation directions for predicting post-editing effort and human evaluation scores, and on some zero-resource directions for post-editing effort.
Translation quality estimation (TQE) is the task of predicting translation quality without reference translations. Due to the enormous cost of creating training data for TQE, only a few translation directions can benefit from supervised training. To address this issue, unsupervised TQE methods have been studied. In this paper, we extensively investigate the usefulness of synthetic TQE data and pre-trained multilingual encoders in unsupervised sentence-level TQE, both of which have been proven effective in the supervised training scenarios. Our experiment on WMT20 and WMT21 datasets revealed that this approach can outperform other unsupervised TQE methods on high- and low-resource translation directions in predicting post-editing effort and human evaluation score, and some zero-resource translation directions in predicting post-editing effort.