Exploiting Linguistic Resources for Neural Machine Translation Using Multi-task Learning
This work addresses the problem of improving translation quality for low-resource language pairs in machine translation, though it is incremental as it builds on existing multi-task learning methods.
The paper tackled the under-explored use of linguistic resources like part-of-speech tags in neural machine translation by applying multi-task learning, resulting in up to a 1.5 BLEU point improvement in German-to-English translation under low-resource conditions.
Linguistic resources such as part-of-speech (POS) tags have been extensively used in statistical machine translation (SMT) frameworks and have yielded better performances. However, usage of such linguistic annotations in neural machine translation (NMT) systems has been left under-explored. In this work, we show that multi-task learning is a successful and a easy approach to introduce an additional knowledge into an end-to-end neural attentional model. By jointly training several natural language processing (NLP) tasks in one system, we are able to leverage common information and improve the performance of the individual task. We analyze the impact of three design decisions in multi-task learning: the tasks used in training, the training schedule, and the degree of parameter sharing across the tasks, which is defined by the network architecture. The experiments are conducted for an German to English translation task. As additional linguistic resources, we exploit POS information and named-entities (NE). Experiments show that the translation quality can be improved by up to 1.5 BLEU points under the low-resource condition. The performance of the POS tagger is also improved using the multi-task learning scheme.