Exploring NLP Benchmarks in an Extremely Low-Resource Setting
It addresses the lack of NLP resources for underrepresented languages like Ladin, providing foundational datasets to support research and applications, though it is incremental in its approach.
This paper tackled the problem of limited NLP datasets for extremely low-resource languages by creating synthetic sentiment analysis and multiple-choice question answering datasets for Ladin, an endangered Romance language, using translation and filtering methods, which improved Italian-Ladin machine translation baselines.
The effectiveness of Large Language Models (LLMs) diminishes for extremely low-resource languages, such as indigenous languages, primarily due to the lack of labeled data. Despite growing interest, the availability of high-quality natural language processing (NLP) datasets for these languages remains limited, making it difficult to develop robust language technologies. This paper addresses such gap by focusing on Ladin, an endangered Romance language, specifically targeting the Val Badia variant. Leveraging a small set of parallel Ladin-Italian sentence pairs, we create synthetic datasets for sentiment analysis and multiple-choice question answering (MCQA) by translating monolingual Italian data. To ensure linguistic quality and reliability, we apply rigorous filtering and back-translation procedures in our method. We further demonstrate that incorporating these synthetic datasets into machine translation training leads to substantial improvements over existing Italian-Ladin translation baselines. Our contributions include the first publicly available sentiment analysis and MCQA datasets for Ladin, establishing foundational resources that can support broader NLP research and downstream applications for this underrepresented language.