Multi-head Span-based Detector for AI-generated Fragments in Scientific Papers
This addresses the challenge of identifying AI-generated content in scientific documents, which is an incremental advancement in a specific domain.
The paper tackles the problem of detecting AI-generated text fragments in scientific papers, achieving a 9% improvement in average macro F1-score from 0.86 to 0.95 on a development set and 0.96 on a closed test set.
This paper describes a system designed to distinguish between AI-generated and human-written scientific excerpts in the DAGPap24 competition hosted within the Fourth Workshop on Scientific Document Processing. In this competition the task is to find artificially generated token-level text fragments in documents of a scientific domain. Our work focuses on the use of a multi-task learning architecture with two heads. The application of this approach is justified by the specificity of the task, where class spans are continuous over several hundred characters. We considered different encoder variations to obtain a state vector for each token in the sequence, as well as a variation in splitting fragments into tokens to further feed into the input of a transform-based encoder. This approach allows us to achieve a 9% quality improvement relative to the baseline solution score on the development set (from 0.86 to 0.95) using the average macro F1-score, as well as a score of 0.96 on a closed test part of the dataset from the competition.