The Evolution of Darija Open Dataset: Introducing Version 2
This dataset addresses the linguistic needs of the Moroccan community by providing resources for Darija NLP, potentially extending to similar dialects, but it is incremental as it builds on an existing dataset.
The paper introduces Version 2 of the Darija Open Dataset (DODa), a large-scale open-source dataset for the Moroccan dialect Darija, containing approximately 100,000 entries with translations, categorizations, and dual-alphabet support to enhance NLP capabilities.
Darija Open Dataset (DODa) represents an open-source project aimed at enhancing Natural Language Processing capabilities for the Moroccan dialect, Darija. With approximately 100,000 entries, DODa stands as the largest collaborative project of its kind for Darija-English translation. The dataset features semantic and syntactic categorizations, variations in spelling, verb conjugations across multiple tenses, as well as tens of thousands of translated sentences. The dataset includes entries written in both Latin and Arabic alphabets, reflecting the linguistic variations and preferences found in different sources and applications. The availability of such dataset is critical for developing applications that can accurately understand and generate Darija, thus supporting the linguistic needs of the Moroccan community and potentially extending to similar dialects in neighboring regions. This paper explores the strategic importance of DODa, its current achievements, and the envisioned future enhancements that will continue to promote its use and expansion in the global NLP landscape.