Adaptive Cross-lingual Text Classification through In-Context One-Shot Demonstrations
This addresses cross-lingual adaptation challenges for NLP practitioners, offering a method that is competitive with fine-tuning using less data, though it appears incremental as it builds on existing in-context tuning techniques.
The paper tackles the problem of performance loss in zero-shot cross-lingual text classification by introducing In-Context Cross-lingual Transfer (IC-XLT), which uses one-shot demonstrations in the target language to adapt models, resulting in improved performance over prompt-based models in zero- and few-shot scenarios.
Zero-Shot Cross-lingual Transfer (ZS-XLT) utilizes a model trained in a source language to make predictions in another language, often with a performance loss. To alleviate this, additional improvements can be achieved through subsequent adaptation using examples in the target language. In this paper, we exploit In-Context Tuning (ICT) for One-Shot Cross-lingual transfer in the classification task by introducing In-Context Cross-lingual Transfer (IC-XLT). The novel concept involves training a model to learn from context examples and subsequently adapting it during inference to a target language by prepending a One-Shot context demonstration in that language. Our results show that IC-XLT successfully leverages target-language examples to improve the cross-lingual capabilities of the evaluated mT5 model, outperforming prompt-based models in the Zero and Few-shot scenarios adapted through fine-tuning. Moreover, we show that when source-language data is limited, the fine-tuning framework employed for IC-XLT performs comparably to prompt-based fine-tuning with significantly more training data in the source language.