Cross-lingual Adaption Model-Agnostic Meta-Learning for Natural Language Understanding
This addresses the problem of improving cross-lingual transfer in NLP for researchers and practitioners, though it appears incremental as it builds on existing meta-learning methods.
The paper tackles the limitation of meta-learning for cross-lingual NLP by proposing XLA-MAML, which enables direct cross-lingual adaptation during meta-training, and shows effectiveness in zero-shot and few-shot experiments on tasks like Natural Language Inference and Question Answering across languages and models.
Meta learning with auxiliary languages has demonstrated promising improvements for cross-lingual natural language processing. However, previous studies sample the meta-training and meta-testing data from the same language, which limits the ability of the model for cross-lingual transfer. In this paper, we propose XLA-MAML, which performs direct cross-lingual adaption in the meta-learning stage. We conduct zero-shot and few-shot experiments on Natural Language Inference and Question Answering. The experimental results demonstrate the effectiveness of our method across different languages, tasks, and pretrained models. We also give analysis on various cross-lingual specific settings for meta-learning including sampling strategy and parallelism.