Progressive Translation: Improving Domain Robustness of Neural Machine Translation with Intermediate Sequences
This work addresses domain robustness for neural machine translation systems, offering incremental improvements with potential benefits in low-resource settings.
The paper tackled the problem of improving domain robustness in Neural Machine Translation by proposing intermediate sequences that introduce an inductive bias to reduce spurious correlations, resulting in reduced hallucinations and enhanced performance in out-of-domain and low-resource scenarios.
Previous studies show that intermediate supervision signals benefit various Natural Language Processing tasks. However, it is not clear whether there exist intermediate signals that benefit Neural Machine Translation (NMT). Borrowing techniques from Statistical Machine Translation, we propose intermediate signals which are intermediate sequences from the "source-like" structure to the "target-like" structure. Such intermediate sequences introduce an inductive bias that reflects a domain-agnostic principle of translation, which reduces spurious correlations that are harmful to out-of-domain generalisation. Furthermore, we introduce a full-permutation multi-task learning to alleviate the spurious causal relations from intermediate sequences to the target, which results from exposure bias. The Minimum Bayes Risk decoding algorithm is used to pick the best candidate translation from all permutations to further improve the performance. Experiments show that the introduced intermediate signals can effectively improve the domain robustness of NMT and reduces the amount of hallucinations on out-of-domain translation. Further analysis shows that our methods are especially promising in low-resource scenarios.