Guidelines for Fine-grained Sentence-level Arabic Readability Annotation
This work addresses the need for standardized readability evaluation in Arabic for educators and researchers, though it is incremental as it builds on existing frameworks.
The paper tackles the problem of fine-grained sentence-level readability assessment in Arabic by presenting annotation guidelines for the Balanced Arabic Readability Evaluation Corpus (BAREC), a large-scale resource with 69,441 sentences labeled across 19 levels, achieving high inter-annotator agreement of 81.8% Quadratic Weighted Kappa.
This paper presents the annotation guidelines of the Balanced Arabic Readability Evaluation Corpus (BAREC), a large-scale resource for fine-grained sentence-level readability assessment in Arabic. BAREC includes 69,441 sentences (1M+ words) labeled across 19 levels, from kindergarten to postgraduate. Based on the Taha/Arabi21 framework, the guidelines were refined through iterative training with native Arabic-speaking educators. We highlight key linguistic, pedagogical, and cognitive factors in determining readability and report high inter-annotator agreement: Quadratic Weighted Kappa 81.8% (substantial/excellent agreement) in the last annotation phase. We also benchmark automatic readability models across multiple classification granularities (19-, 7-, 5-, and 3-level). The corpus and guidelines are publicly available.