GATech at AbjadGenEval Shared Task: Multilingual Embeddings for Arabic Machine-Generated Text Classification
This work addresses the challenge of machine-generated text detection in Arabic, which is important for content moderation and security, but it is incremental as it applies existing methods to a new dataset.
The researchers tackled the problem of detecting AI-generated Arabic text by fine-tuning a multilingual E5-large encoder for binary classification, achieving an F1 score of 0.75 on the test set using simple mean pooling.
We present our approach to the AbjadGenEval shared task on detecting AI-generated Arabic text. We fine-tuned the multilingual E5-large encoder for binary classification, and we explored several pooling strategies to pool token representations, including weighted layer pooling, multi-head attention pooling, and gated fusion. Interestingly, none of these outperformed simple mean pooling, which achieved an F1 of 0.75 on the test set. We believe this is because complex pooling methods introduce additional parameters that need more data to train properly, whereas mean pooling offers a stable baseline that generalizes well even with limited examples. We also observe a clear pattern in the data: human-written texts tend to be significantly longer than machine-generated ones.