A Collection of Question Answering Datasets for Norwegian
This provides new benchmark datasets for Norwegian NLP, though it is incremental as it adapts existing QA formats to a new language.
The paper introduces four new Norwegian question answering datasets covering over 10k question-answer pairs across both written standards, and evaluates 11 language models showing they perform better in Bokmål than Nynorsk, struggle with commonsense reasoning, and often generate untruthful answers.
This paper introduces a new suite of question answering datasets for Norwegian; NorOpenBookQA, NorCommonSenseQA, NorTruthfulQA, and NRK-Quiz-QA. The data covers a wide range of skills and knowledge domains, including world knowledge, commonsense reasoning, truthfulness, and knowledge about Norway. Covering both of the written standards of Norwegian - Bokmål and Nynorsk - our datasets comprise over 10k question-answer pairs, created by native speakers. We detail our dataset creation approach and present the results of evaluating 11 language models (LMs) in zero- and few-shot regimes. Most LMs perform better in Bokmål than Nynorsk, struggle most with commonsense reasoning, and are often untruthful in generating answers to questions. All our datasets and annotation materials are publicly available.