Multi-lingual and Multi-cultural Figurative Language Understanding
This work addresses the lack of cultural and linguistic diversity in NLP datasets for figurative language, which is crucial for improving language models' applicability across different societies, though it is incremental as it extends existing dataset creation efforts to new languages.
The authors tackled the problem of figurative language understanding in NLP by creating a multilingual dataset for seven diverse languages, revealing that cultural and regional concepts heavily influence figurative expressions and showing that multilingual language models perform significantly worse on these languages compared to English.
Figurative language permeates human communication, but at the same time is relatively understudied in NLP. Datasets have been created in English to accelerate progress towards measuring and improving figurative language processing in language models (LMs). However, the use of figurative language is an expression of our cultural and societal experiences, making it difficult for these phrases to be universally applicable. In this work, we create a figurative language inference dataset, \datasetname, for seven diverse languages associated with a variety of cultures: Hindi, Indonesian, Javanese, Kannada, Sundanese, Swahili and Yoruba. Our dataset reveals that each language relies on cultural and regional concepts for figurative expressions, with the highest overlap between languages originating from the same region. We assess multilingual LMs' abilities to interpret figurative language in zero-shot and few-shot settings. All languages exhibit a significant deficiency compared to English, with variations in performance reflecting the availability of pre-training and fine-tuning data, emphasizing the need for LMs to be exposed to a broader range of linguistic and cultural variation during training.