Incorporating Structural Alignment Biases into an Attentional Neural Translation Model
This work addresses the need for more robust translation models in low-resource languages, though it is incremental as it builds on existing attentional neural translation models.
The authors tackled the problem of neural machine translation models lacking key inductive biases from traditional alignment models by incorporating structural biases like positional bias and fertility into an attentional neural translation model. They achieved improvements over baseline attentional and phrase-based models across several language pairs in low-resource settings.
Neural encoder-decoder models of machine translation have achieved impressive results, rivalling traditional translation models. However their modelling formulation is overly simplistic, and omits several key inductive biases built into traditional models. In this paper we extend the attentional neural translation model to include structural biases from word based alignment models, including positional bias, Markov conditioning, fertility and agreement over translation directions. We show improvements over a baseline attentional model and standard phrase-based model over several language pairs, evaluating on difficult languages in a low resource setting.