Triospect: A Three-Dimensional Framework for Robust Statistical AI-Generated Text Detection Against Diverse Attacks
This work addresses the vulnerability of AI-generated text detectors to adversarial attacks, providing a more reliable detection method for security-critical applications.
The paper proposes Triospect, a three-dimensional framework for detecting AI-generated text that is robust against diverse attacks. It achieves a 22.3% improvement in AUROC and 13% in TPR01 on the Humanize-16K dataset, and 9.1% and 22% on the adversarial RAID dataset.
Existing AI-generated text detectors are vulnerable to attacks that manipulate textual characteristics. In this study, we propose a novel Triospect Detection Framework by using additional perspectives of content (core ideas) and expression (stylistic elements) within a given text. Experiments on two benchmarks involving 17 attacks, 12 domains, and 17 source models demonstrate that Triospect is robust against these attacks. It improves the strong baseline by a significant margin of 22.3% (AUROC) and 13% (TPR01) on the Humanize-16K after-attack subset, and by 9.1% (AUROC) and 22% (TPR01) on the adversarial RAID. This framework marks a pioneering effort in statistical methods to enhance detection reliability against attacks. We release our data and code at https://github.com/baoguangsheng/triospect.