Variable-length Neural Interlingua Representations for Zero-shot Neural Machine Translation
This addresses the problem of improving zero-shot translation flexibility for multilingual models, but it is incremental as it builds on existing neural interlingua methods.
The paper tackled the limitation of fixed-length neural interlingua representations in multilingual neural machine translation by introducing variable-length representations, resulting in stable convergence and superior zero-shot translation results on datasets like OPUS, IWSLT, and Europarl, though it showed suboptimal efficacy for certain source languages.
The language-independency of encoded representations within multilingual neural machine translation (MNMT) models is crucial for their generalization ability on zero-shot translation. Neural interlingua representations have been shown as an effective method for achieving this. However, fixed-length neural interlingua representations introduced in previous work can limit its flexibility and representation ability. In this study, we introduce a novel method to enhance neural interlingua representations by making their length variable, thereby overcoming the constraint of fixed-length neural interlingua representations. Our empirical results on zero-shot translation on OPUS, IWSLT, and Europarl datasets demonstrate stable model convergence and superior zero-shot translation results compared to fixed-length neural interlingua representations. However, our analysis reveals the suboptimal efficacy of our approach in translating from certain source languages, wherein we pinpoint the defective model component in our proposed method.