Translation Artifacts in Cross-lingual Transfer Learning
This addresses a critical issue for researchers and practitioners in multilingual NLP, as it reveals artifacts that affect model performance and requires reconsideration of previous findings, though it is incremental in improving specific benchmarks.
The paper tackles the problem of translation artifacts in cross-lingual transfer learning, showing that translation processes introduce subtle artifacts that notably impact existing models, such as reducing lexical overlap in natural language inference. It improves state-of-the-art in XNLI by 4.3 points for translate-test and 2.8 points for zero-shot approaches.
Both human and machine translation play a central role in cross-lingual transfer learning: many multilingual datasets have been created through professional translation services, and using machine translation to translate either the test set or the training set is a widely used transfer technique. In this paper, we show that such translation process can introduce subtle artifacts that have a notable impact in existing cross-lingual models. For instance, in natural language inference, translating the premise and the hypothesis independently can reduce the lexical overlap between them, which current models are highly sensitive to. We show that some previous findings in cross-lingual transfer learning need to be reconsidered in the light of this phenomenon. Based on the gained insights, we also improve the state-of-the-art in XNLI for the translate-test and zero-shot approaches by 4.3 and 2.8 points, respectively.