Dual Reconstruction: a Unifying Objective for Semi-Supervised Neural Machine Translation
This work addresses the challenge of effectively incorporating monolingual data in machine translation for researchers and practitioners, but it is incremental as it compares existing methods rather than introducing a new paradigm.
The paper tackled the problem of unifying and comparing two semi-supervised methods for neural machine translation, Iterative Back-Translation and Dual Learning, by introducing a dual reconstruction objective. The result showed that Iterative Back-Translation is more effective than Dual Learning in German-English and Turkish-English tasks, as supported by theoretical analysis and empirical study.
While Iterative Back-Translation and Dual Learning effectively incorporate monolingual training data in neural machine translation, they use different objectives and heuristic gradient approximation strategies, and have not been extensively compared. We introduce a novel dual reconstruction objective that provides a unified view of Iterative Back-Translation and Dual Learning. It motivates a theoretical analysis and controlled empirical study on German-English and Turkish-English tasks, which both suggest that Iterative Back-Translation is more effective than Dual Learning despite its relative simplicity.