IndicXNLI: Evaluating Multilingual Inference for Indian Languages
This provides a new benchmark for evaluating multilingual inference in Indian languages, addressing a gap in Indic NLP, though it is incremental as it builds on existing XNLI data.
The authors tackled the lack of benchmark datasets for natural language understanding in Indian languages by introducing IndicXNLI, a high-quality machine-translated NLI dataset for 11 languages, and analyzed cross-lingual transfer techniques using pre-trained models to gain insights into their behavior.
While Indic NLP has made rapid advances recently in terms of the availability of corpora and pre-trained models, benchmark datasets on standard NLU tasks are limited. To this end, we introduce IndicXNLI, an NLI dataset for 11 Indic languages. It has been created by high-quality machine translation of the original English XNLI dataset and our analysis attests to the quality of IndicXNLI. By finetuning different pre-trained LMs on this IndicXNLI, we analyze various cross-lingual transfer techniques with respect to the impact of the choice of language models, languages, multi-linguality, mix-language input, etc. These experiments provide us with useful insights into the behaviour of pre-trained models for a diverse set of languages.