Konooz: Multi-domain Multi-dialect Corpus for Named Entity Recognition
This addresses the challenge of resource scarcity and model robustness for Arabic NLP, but is incremental as it primarily benchmarks existing methods on new data.
The authors tackled the problem of evaluating Arabic Named Entity Recognition (NER) models across diverse dialects and domains by introducing Konooz, a multi-domain multi-dialect corpus, and found that existing models experienced performance drops of up to 38% compared to in-distribution data.
We introduce Konooz, a novel multi-dimensional corpus covering 16 Arabic dialects across 10 domains, resulting in 160 distinct corpora. The corpus comprises about 777k tokens, carefully collected and manually annotated with 21 entity types using both nested and flat annotation schemes - using the Wojood guidelines. While Konooz is useful for various NLP tasks like domain adaptation and transfer learning, this paper primarily focuses on benchmarking existing Arabic Named Entity Recognition (NER) models, especially cross-domain and cross-dialect model performance. Our benchmarking of four Arabic NER models using Konooz reveals a significant drop in performance of up to 38% when compared to the in-distribution data. Furthermore, we present an in-depth analysis of domain and dialect divergence and the impact of resource scarcity. We also measured the overlap between domains and dialects using the Maximum Mean Discrepancy (MMD) metric, and illustrated why certain NER models perform better on specific dialects and domains. Konooz is open-source and publicly available at https://sina.birzeit.edu/wojood/#download