Novel Applications of Factored Neural Machine Translation
This work addresses incremental improvements in neural machine translation efficiency and performance for researchers and practitioners in computational linguistics.
The paper tackled the problem of extending target factors in neural machine translation beyond predicting word lemmas and inflections by introducing three novel applications: predicting word case separately, eliminating subword joining markers, and predicting special tokens in operation sequence NMT. Results showed that integration of these tasks was at least as good as standard NMT, with a significant BLEU improvement for operation sequence NMT due to reduced output sequence length.
In this work, we explore the usefulness of target factors in neural machine translation (NMT) beyond their original purpose of predicting word lemmas and their inflections, as proposed by Garcìa-Martìnez et al., 2016. For this, we introduce three novel applications of the factored output architecture: In the first one, we use a factor to explicitly predict the word case separately from the target word itself. This allows for information to be shared between different casing variants of a word. In a second task, we use a factor to predict when two consecutive subwords have to be joined, eliminating the need for target subword joining markers. The third task is the prediction of special tokens of the operation sequence NMT model (OSNMT) of Stahlberg et al., 2018. Automatic evaluation on English-to-German and English-to-Turkish tasks showed that integration of such auxiliary prediction tasks into NMT is at least as good as the standard NMT approach. For the OSNMT, we observed a significant improvement in BLEU over the baseline OSNMT implementation due to a reduced output sequence length that resulted from the introduction of the target factors.