Zero-Shot Translation Quality Estimation with Explicit Cross-Lingual Patterns
This work addresses translation quality estimation for machine translation systems, offering a zero-shot approach that is incremental but reduces reliance on labeled data.
The paper tackles the mismatching issue when using BERTScore for translation quality estimation by proposing a zero-shot model with explicit cross-lingual patterns, achieving comparable performance to supervised methods and outperforming them in 2 out of 6 directions.
This paper describes our submission of the WMT 2020 Shared Task on Sentence Level Direct Assessment, Quality Estimation (QE). In this study, we empirically reveal the \textit{mismatching issue} when directly adopting BERTScore to QE. Specifically, there exist lots of mismatching errors between the source sentence and translated candidate sentence with token pairwise similarity. In response to this issue, we propose to expose explicit cross-lingual patterns, \textit{e.g.} word alignments and generation score, to our proposed zero-shot models. Experiments show that our proposed QE model with explicit cross-lingual patterns could alleviate the mismatching issue, thereby improving the performance. Encouragingly, our zero-shot QE method could achieve comparable performance with supervised QE method, and even outperforms the supervised counterpart on 2 out of 6 directions. We expect our work could shed light on the zero-shot QE model improvement.