Improving Bidirectional Decoding with Dynamic Target Semantics in Neural Machine Translation
This work addresses translation quality issues in neural machine translation for language pairs like Chinese-English and English-Romanian, representing an incremental improvement over existing bidirectional decoding methods.
The paper tackles the problem of unbalanced translations in neural machine translation by proposing a Dynamic Interaction Module (DIM) to dynamically exploit target semantics from right-to-left translations, achieving significant improvements over baseline systems on NIST Chinese-English and WMT'16 English-Romanian tasks, with results comparable to state-of-the-art Transformer models using fewer parameters.
Generally, Neural Machine Translation models generate target words in a left-to-right (L2R) manner and fail to exploit any future (right) semantics information, which usually produces an unbalanced translation. Recent works attempt to utilize the right-to-left (R2L) decoder in bidirectional decoding to alleviate this problem. In this paper, we propose a novel \textbf{D}ynamic \textbf{I}nteraction \textbf{M}odule (\textbf{DIM}) to dynamically exploit target semantics from R2L translation for enhancing the L2R translation quality. Different from other bidirectional decoding approaches, DIM firstly extracts helpful target information through addressing and reading operations, then updates target semantics for tracking the interactive history. Additionally, we further introduce an \textbf{agreement regularization} term into the training objective to narrow the gap between L2R and R2L translations. Experimental results on NIST Chinese$\Rightarrow$English and WMT'16 English$\Rightarrow$Romanian translation tasks show that our system achieves significant improvements over baseline systems, which also reaches comparable results compared to the state-of-the-art Transformer model with much fewer parameters of it.