AraLegal-BERT: A pretrained language model for Arabic Legal text
This work addresses the need for domain-specific NLP solutions in Arabic legal practice, though it is incremental as it adapts existing BERT methods to a new domain.
The authors tackled the problem of applying BERT to Arabic legal text by introducing AraLegal-BERT, a pretrained model customized for this domain, which achieved better accuracy than general BERT variants in three NLU tasks.
The effectiveness of the BERT model on multiple linguistic tasks has been well documented. On the other hand, its potentials for narrow and specific domains such as Legal, have not been fully explored. In this paper, we examine how BERT can be used in the Arabic legal domain and try customizing this language model for several downstream tasks using several different domain-relevant training and testing datasets to train BERT from scratch. We introduce the AraLegal-BERT, a bidirectional encoder Transformer-based model that have been thoroughly tested and carefully optimized with the goal to amplify the impact of NLP-driven solution concerning jurisprudence, legal documents, and legal practice. We fine-tuned AraLegal-BERT and evaluated it against three BERT variations for Arabic language in three natural languages understanding (NLU) tasks. The results show that the base version of AraLegal-BERT achieve better accuracy than the general and original BERT over the Legal text.