Context-Aware Monolingual Human Evaluation of Machine Translation
This addresses the need for efficient evaluation methods in machine translation, particularly when source texts are unavailable, though it is incremental as it builds on existing evaluation practices.
The paper tackled the problem of evaluating machine translation without source text by comparing context-aware monolingual human evaluations to bilingual ones, finding that monolingual evaluations achieve comparable outcomes, with professional translators providing ratings and error annotations.
This paper explores the potential of context-aware monolingual human evaluation for assessing machine translation (MT) when no source is given for reference. To this end, we compare monolingual with bilingual evaluations (with source text), under two scenarios: the evaluation of a single MT system, and the comparative evaluation of pairwise MT systems. Four professional translators performed both monolingual and bilingual evaluations by assigning ratings and annotating errors, and providing feedback on their experience. Our findings suggest that context-aware monolingual human evaluation achieves comparable outcomes to human bilingual evaluations, and suggest the feasibility and potential of monolingual evaluation as an efficient approach to assessing MT.