Learning to Refine Source Representations for Neural Machine Translation
This addresses translation quality issues for NMT users by introducing a novel framework that mimics human incremental refinement, though it appears incremental as it builds on standard encoder-decoder architectures.
The paper tackles the problem of ambiguous source sentences in neural machine translation by proposing an encoder-refiner-decoder framework that dynamically refines source representations based on target-side information during decoding, with results showing significant and consistent performance improvements on Chinese-English and English-German translation tasks.
Neural machine translation (NMT) models generally adopt an encoder-decoder architecture for modeling the entire translation process. The encoder summarizes the representation of input sentence from scratch, which is potentially a problem if the sentence is ambiguous. When translating a text, humans often create an initial understanding of the source sentence and then incrementally refine it along the translation on the target side. Starting from this intuition, we propose a novel encoder-refiner-decoder framework, which dynamically refines the source representations based on the generated target-side information at each decoding step. Since the refining operations are time-consuming, we propose a strategy, leveraging the power of reinforcement learning models, to decide when to refine at specific decoding steps. Experimental results on both Chinese-English and English-German translation tasks show that the proposed approach significantly and consistently improves translation performance over the standard encoder-decoder framework. Furthermore, when refining strategy is applied, results still show reasonable improvement over the baseline without much decrease in decoding speed.