Almost AI, Almost Human: The Challenge of Detecting AI-Polished Writing
This addresses the challenge of false plagiarism accusations and misleading AI prevalence claims for online content users, though it is incremental in improving detection methodologies.
The study tackled the problem of AI-polished text detection, where human-written content is subtly refined by AI tools, and found that current detectors frequently misclassify minimally polished text as AI-generated, struggle to differentiate AI involvement levels, and show biases against older and smaller models.
The growing use of large language models (LLMs) for text generation has led to widespread concerns about AI-generated content detection. However, an overlooked challenge is AI-polished text, where human-written content undergoes subtle refinements using AI tools. This raises a critical question: should minimally polished text be classified as AI-generated? Such classification can lead to false plagiarism accusations and misleading claims about AI prevalence in online content. In this study, we systematically evaluate twelve state-of-the-art AI-text detectors using our AI-Polished-Text Evaluation (APT-Eval) dataset, which contains 14.7K samples refined at varying AI-involvement levels. Our findings reveal that detectors frequently flag even minimally polished text as AI-generated, struggle to differentiate between degrees of AI involvement, and exhibit biases against older and smaller models. These limitations highlight the urgent need for more nuanced detection methodologies.