Scaling up COMETKIWI: Unbabel-IST 2023 Submission for the Quality Estimation Shared Task
This work addresses the need for accurate quality estimation in machine translation, particularly for multilingual applications, though it is incremental as it builds on an existing model.
The authors tackled the problem of quality estimation in machine translation by scaling up the COMETKIWI-22 model, achieving state-of-the-art performance across word-, span-, and sentence-level tasks with improvements of up to 10 Spearman points in correlation with human judgments.
We present the joint contribution of Unbabel and Instituto Superior Técnico to the WMT 2023 Shared Task on Quality Estimation (QE). Our team participated on all tasks: sentence- and word-level quality prediction (task 1) and fine-grained error span detection (task 2). For all tasks, we build on the COMETKIWI-22 model (Rei et al., 2022b). Our multilingual approaches are ranked first for all tasks, reaching state-of-the-art performance for quality estimation at word-, span- and sentence-level granularity. Compared to the previous state-of-the-art COMETKIWI-22, we show large improvements in correlation with human judgements (up to 10 Spearman points). Moreover, we surpass the second-best multilingual submission to the shared-task with up to 3.8 absolute points.