Rethinking Translation Memory Augmented Neural Machine Translation
This addresses the problem of inconsistent performance in machine translation for researchers and practitioners, but it is incremental as it builds on existing TM-augmented NMT methods.
The paper tackles the contradictory performance of translation memory augmented neural machine translation (TM-augmented NMT) by analyzing it through a probabilistic view and variance-bias decomposition, showing it has lower bias but higher variance, and proposes a model that achieves consistent gains over conventional and existing TM-augmented NMT across low-resource, plug-and-play, and high-resource scenarios.
This paper rethinks translation memory augmented neural machine translation (TM-augmented NMT) from two perspectives, i.e., a probabilistic view of retrieval and the variance-bias decomposition principle. The finding demonstrates that TM-augmented NMT is good at the ability of fitting data (i.e., lower bias) but is more sensitive to the fluctuations in the training data (i.e., higher variance), which provides an explanation to a recently reported contradictory phenomenon on the same translation task: TM-augmented NMT substantially advances vanilla NMT under the high-resource scenario whereas it fails under the low-resource scenario. Then we propose a simple yet effective TM-augmented NMT model to promote the variance and address the contradictory phenomenon. Extensive experiments show that the proposed TM-augmented NMT achieves consistent gains over both conventional NMT and existing TM-augmented NMT under two variance-preferable (low-resource and plug-and-play) scenarios as well as the high-resource scenario.