Efficient Bidirectional Neural Machine Translation
This addresses translation quality issues for NLP practitioners, offering a more efficient alternative to ensemble methods.
The paper tackled the problem of error propagation in neural machine translation by proposing an efficient bidirectional generation method using a single encoder-decoder, which improved translation quality over unidirectional approaches and saved 50% model parameters and 40% training time compared to ensemble methods.
The encoder-decoder based neural machine translation usually generates a target sequence token by token from left to right. Due to error propagation, the tokens in the right side of the generated sequence are usually of poorer quality than those in the left side. In this paper, we propose an efficient method to generate a sequence in both left-to-right and right-to-left manners using a single encoder and decoder, combining the advantages of both generation directions. Experiments on three translation tasks show that our method achieves significant improvements over conventional unidirectional approach. Compared with ensemble methods that train and combine two models with different generation directions, our method saves 50% model parameters and about 40% training time, and also improve inference speed.