Leveraging BERT Language Model for Arabic Long Document Classification
This addresses the computational challenges of long document classification for Arabic speakers in fields like law and medicine, but it is incremental as it builds on existing methods.
The authors tackled the problem of classifying long Arabic documents by proposing two simple models, which outperformed fine-tuned Longformer and RoBERTa models on two datasets.
Given the number of Arabic speakers worldwide and the notably large amount of content in the web today in some fields such as law, medicine, or even news, documents of considerable length are produced regularly. Classifying those documents using traditional learning models is often impractical since extended length of the documents increases computational requirements to an unsustainable level. Thus, it is necessary to customize these models specifically for long textual documents. In this paper we propose two simple but effective models to classify long length Arabic documents. We also fine-tune two different models-namely, Longformer and RoBERT, for the same task and compare their results to our models. Both of our models outperform the Longformer and RoBERT in this task over two different datasets.