Morphemes Without Borders: Evaluating Root-Pattern Morphology in Arabic Tokenizers and LLMs
This addresses the problem of understanding morphological representation in LLMs for Arabic, with incremental insights into tokenization effects.
The study evaluated how well large language models (LLMs) and their tokenizers handle Arabic root-pattern morphology, finding that tokenizer morphological alignment is neither necessary nor sufficient for morphological generation, questioning its role in downstream performance.
This work investigates how effectively large language models (LLMs) and their tokenization schemes represent and generate Arabic root-pattern morphology, probing whether they capture genuine morphological structure or rely on surface memorization. Arabic morphological system provides a rich testbed for analyzing how LLMs handle complex, non-concatenative forms and how tokenization choices influence this process. Our study begins with an evaluation of morphological fidelity across Arabic and multilingual tokenizers against gold-standard segmentation, followed by an analysis of LLM performance in productive root-pattern generation using a newly developed test set. Our findings across seven Arabic-centric and multilingual LLMs and their respective tokenizers reveal that tokenizer morphological alignment is not necessary nor sufficient for morphological generation, which questions the role of morphological tokenization in downstream performance.