Automatic Discrimination of Human and Neural Machine Translation in Multilingual Scenarios
This work addresses the challenge of detecting machine-generated translations for applications in quality control and content moderation, but it is incremental as it builds on existing methods by extending them to multilingual scenarios.
The paper tackled the problem of automatically distinguishing between human and machine translations in a multilingual context, showing that classifiers trained on data from one source language can generalize to others and that incorporating source text improves accuracy and robustness.
We tackle the task of automatically discriminating between human and machine translations. As opposed to most previous work, we perform experiments in a multilingual setting, considering multiple languages and multilingual pretrained language models. We show that a classifier trained on parallel data with a single source language (in our case German-English) can still perform well on English translations that come from different source languages, even when the machine translations were produced by other systems than the one it was trained on. Additionally, we demonstrate that incorporating the source text in the input of a multilingual classifier improves (i) its accuracy and (ii) its robustness on cross-system evaluation, compared to a monolingual classifier. Furthermore, we find that using training data from multiple source languages (German, Russian, and Chinese) tends to improve the accuracy of both monolingual and multilingual classifiers. Finally, we show that bilingual classifiers and classifiers trained on multiple source languages benefit from being trained on longer text sequences, rather than on sentences.