Repairing Pronouns in Translation with BERT-Based Post-Editing
This addresses pronoun translation issues in NMT, which impair intelligibility and reinforce gender bias, but the approach is incremental as it builds on existing BERT and NMT methods.
The paper tackles the problem of pronoun errors in Neural Machine Translation (NMT), which can account for over half of errors in some domains and significantly impact perceived quality, by proposing a BERT-based post-editing method that fine-tunes BERT for pronoun prediction to repair translations, with initial results showing significant improvements in a small number of Japanese-English translations according to human evaluators.
Pronouns are important determinants of a text's meaning but difficult to translate. This is because pronoun choice can depend on entities described in previous sentences, and in some languages pronouns may be dropped when the referent is inferrable from the context. These issues can lead Neural Machine Translation (NMT) systems to make critical errors on pronouns that impair intelligibility and even reinforce gender bias. We investigate the severity of this pronoun issue, showing that (1) in some domains, pronoun choice can account for more than half of a NMT systems' errors, and (2) pronouns have a disproportionately large impact on perceived translation quality. We then investigate a possible solution: fine-tuning BERT on a pronoun prediction task using chunks of source-side sentences, then using the resulting classifier to repair the translations of an existing NMT model. We offer an initial case study of this approach for the Japanese-English language pair, observing that a small number of translations are significantly improved according to human evaluators.