BUSTED at AraGenEval Shared Task: A Comparative Study of Transformer-Based Models for Arabic AI-Generated Text Detection
This work addresses AI-generated text detection for Arabic language applications, but it is incremental as it applies existing methods to a new dataset without introducing novel techniques.
The paper tackled the problem of detecting AI-generated text in Arabic by fine-tuning transformer models, finding that the multilingual XLM-RoBERTa model achieved the highest performance with an F1 score of 0.7701, outperforming specialized Arabic models.
This paper details our submission to the AraGenEval Shared Task on Arabic AI-generated text detection, where our team, BUSTED, secured 5th place. We investigated the effectiveness of three pre-trained transformer models: AraELECTRA, CAMeLBERT, and XLM-RoBERTa. Our approach involved fine-tuning each model on the provided dataset for a binary classification task. Our findings revealed a surprising result: the multilingual XLM-RoBERTa model achieved the highest performance with an F1 score of 0.7701, outperforming the specialized Arabic models. This work underscores the complexities of AI-generated text detection and highlights the strong generalization capabilities of multilingual models.