Learning to Rewrite: Generalized LLM-Generated Text Detection
This addresses the risk of LLM-generated disinformation by improving detection generalization across domains, though it is incremental as it builds on existing rewriting insights.
The paper tackles the problem of detecting AI-generated text by introducing Learning2Rewrite, a framework that trains LLMs to minimize alterations on AI-generated content, achieving up to 23.04% higher AUROC in in-distribution tests, 37.26% in out-of-distribution tests, and 48.66% under adversarial attacks compared to state-of-the-art methods.
Large language models (LLMs) present significant risks when used to generate non-factual content and spread disinformation at scale. Detecting such LLM-generated content is crucial, yet current detectors often struggle to generalize in open-world contexts. We introduce Learning2Rewrite, a novel framework for detecting AI-generated text with exceptional generalization to unseen domains. Our method leverages the insight that LLMs inherently modify AI-generated content less than human-written text when tasked with rewriting. By training LLMs to minimize alterations on AI-generated inputs, we amplify this disparity, yielding a more distinguishable and generalizable edit distance across diverse text distributions. Extensive experiments on data from 21 independent domains and four major LLMs (GPT-3.5, GPT-4, Gemini, and Llama-3) demonstrate that our detector outperforms state-of-the-art detection methods by up to 23.04% in AUROC for in-distribution tests, 37.26% for out-of-distribution tests, and 48.66% under adversarial attacks. Our unique training objective ensures better generalizability compared to directly training for classification, when leveraging the same amount of parameters. Our findings suggest that reinforcing LLMs' inherent rewriting tendencies offers a robust and scalable solution for detecting AI-generated text.