Neural Machine Translation with Reconstruction
This addresses a major drawback in NMT for machine translation applications, though it appears incremental as it builds on existing NMT models.
The paper tackles the problem of inadequate translations in Neural Machine Translation (NMT) by proposing an encoder-decoder-reconstructor framework, which significantly improves adequacy and achieves superior translation results over state-of-the-art systems.
Although end-to-end Neural Machine Translation (NMT) has achieved remarkable progress in the past two years, it suffers from a major drawback: translations generated by NMT systems often lack of adequacy. It has been widely observed that NMT tends to repeatedly translate some source words while mistakenly ignoring other words. To alleviate this problem, we propose a novel encoder-decoder-reconstructor framework for NMT. The reconstructor, incorporated into the NMT model, manages to reconstruct the input source sentence from the hidden layer of the output target sentence, to ensure that the information in the source side is transformed to the target side as much as possible. Experiments show that the proposed framework significantly improves the adequacy of NMT output and achieves superior translation result over state-of-the-art NMT and statistical MT systems.