UPB at IberLEF-2023 AuTexTification: Detection of Machine-Generated Text using Transformer Ensembles
This work addresses the problem of identifying AI-generated content for tasks like content moderation, but it is incremental as it applies existing methods to a new shared task.
The paper tackled detecting machine-generated text in English and Spanish across domains like legal and social media, achieving macro F1-scores of 66.63% and 67.10% using transformer ensembles.
This paper describes the solutions submitted by the UPB team to the AuTexTification shared task, featured as part of IberLEF-2023. Our team participated in the first subtask, identifying text documents produced by large language models instead of humans. The organizers provided a bilingual dataset for this subtask, comprising English and Spanish texts covering multiple domains, such as legal texts, social media posts, and how-to articles. We experimented mostly with deep learning models based on Transformers, as well as training techniques such as multi-task learning and virtual adversarial training to obtain better results. We submitted three runs, two of which consisted of ensemble models. Our best-performing model achieved macro F1-scores of 66.63% on the English dataset and 67.10% on the Spanish dataset.