Discrete and Soft Prompting for Multilingual Models
This addresses the challenge of efficient multilingual adaptation for pretrained language models, though it is incremental as it extends known prompting methods to multilingual settings.
The paper tackled the problem of few-shot learning in multilingual natural language inference, showing that discrete and soft prompting outperform finetuning in crosslingual transfer and in-language training, with concrete improvements such as 38.79% accuracy versus 33.74% for finetuning using 48 English examples.
It has been shown for English that discrete and soft prompting perform strongly in few-shot learning with pretrained language models (PLMs). In this paper, we show that discrete and soft prompting perform better than finetuning in multilingual cases: Crosslingual transfer and in-language training of multilingual natural language inference. For example, with 48 English training examples, finetuning obtains 33.74% accuracy in crosslingual transfer, barely surpassing the majority baseline (33.33%). In contrast, discrete and soft prompting outperform finetuning, achieving 36.43% and 38.79%. We also demonstrate good performance of prompting with training data in multiple languages other than English.