UM4: Unified Multilingual Multiple Teacher-Student Model for Zero-Resource Neural Machine Translation
This addresses the problem of translation between languages without parallel data for multilingual NLP applications, representing a strong incremental improvement over existing pivot-based and multilingual methods.
The paper tackles zero-resource neural machine translation where parallel corpora are unavailable, proposing UM4 which unifies multiple teacher-student models to guide translation; it demonstrates significant outperformance over previous methods on the WMT benchmark across 72 translation directions.
Most translation tasks among languages belong to the zero-resource translation problem where parallel corpora are unavailable. Multilingual neural machine translation (MNMT) enables one-pass translation using shared semantic space for all languages compared to the two-pass pivot translation but often underperforms the pivot-based method. In this paper, we propose a novel method, named as Unified Multilingual Multiple teacher-student Model for NMT (UM4). Our method unifies source-teacher, target-teacher, and pivot-teacher models to guide the student model for the zero-resource translation. The source teacher and target teacher force the student to learn the direct source to target translation by the distilled knowledge on both source and target sides. The monolingual corpus is further leveraged by the pivot-teacher model to enhance the student model. Experimental results demonstrate that our model of 72 directions significantly outperforms previous methods on the WMT benchmark.