Neural Machine Translation from Simplified Translations
This work addresses the challenge of translation complexity for machine translation systems, presenting an incremental improvement based on knowledge distillation.
The paper tackles the problem of improving neural machine translation by using simplified translations to enhance model learning, showing that training on simplified bi-text outperforms the baseline system and achieves further gains when combined with reference translations.
Text simplification aims at reducing the lexical, grammatical and structural complexity of a text while keeping the same meaning. In the context of machine translation, we introduce the idea of simplified translations in order to boost the learning ability of deep neural translation models. We conduct preliminary experiments showing that translation complexity is actually reduced in a translation of a source bi-text compared to the target reference of the bi-text while using a neural machine translation (NMT) system learned on the exact same bi-text. Based on knowledge distillation idea, we then train an NMT system using the simplified bi-text, and show that it outperforms the initial system that was built over the reference data set. Performance is further boosted when both reference and automatic translations are used to learn the network. We perform an elementary analysis of the translated corpus and report accuracy results of the proposed approach on English-to-French and English-to-German translation tasks.