Machine Translation Models are Zero-Shot Detectors of Translation Direction
This work addresses a forensic and machine translation evaluation problem, but it is incremental as it builds on known translationese effects.
The paper tackles the problem of detecting the translation direction of parallel text by proposing an unsupervised approach based on the hypothesis that translation probability is higher from original to translation. It achieves document-level accuracies of 82-96% for NMT-produced translations and 60-81% for human translations across 20 high-resource language pairs.
Detecting the translation direction of parallel text has applications for machine translation training and evaluation, but also has forensic applications such as resolving plagiarism or forgery allegations. In this work, we explore an unsupervised approach to translation direction detection based on the simple hypothesis that $p(\text{translation}|\text{original})>p(\text{original}|\text{translation})$, motivated by the well-known simplification effect in translationese or machine-translationese. In experiments with massively multilingual machine translation models across 20 translation directions, we confirm the effectiveness of the approach for high-resource language pairs, achieving document-level accuracies of 82--96% for NMT-produced translations, and 60--81% for human translations, depending on the model used. Code and demo are available at https://github.com/ZurichNLP/translation-direction-detection