Detecting over/under-translation errors for determining adequacy in human translations
This addresses the need for automated error detection in human translation pipelines, particularly for subtitles, though it is incremental as it builds on existing classification methods.
The paper tackled the problem of detecting over- and under-translation errors in human translations, specifically for video subtitles, by developing a model trained on synthesized data without reference translations, achieving 89.3% accuracy on human-annotated evaluation data across 8 languages.
We present a novel approach to detecting over and under translations (OT/UT) as part of adequacy error checks in translation evaluation. We do not restrict ourselves to machine translation (MT) outputs and specifically target applications with human generated translation pipeline. The goal of our system is to identify OT/UT errors from human translated video subtitles with high error recall. We achieve this without reference translations by learning a model on synthesized training data. We compare various classification networks that we trained on embeddings from pre-trained language model with our best hybrid network of GRU + CNN achieving 89.3% accuracy on high-quality human-annotated evaluation data in 8 languages.