EAGLE: A Domain Generalization Framework for AI-generated Text Detection
This addresses the challenge of scalable and practical AI-generated text detection for users and platforms needing to identify content from evolving LLMs, representing a significant but incremental improvement over existing supervised methods.
The paper tackles the problem of detecting AI-generated text from new, unseen language models without requiring labeled data for each new model, proposing EAGLE, a domain generalization framework that achieves detection scores within 4.7% of a fully supervised detector for models like GPT-4 and Claude.
With the advancement in capabilities of Large Language Models (LLMs), one major step in the responsible and safe use of such LLMs is to be able to detect text generated by these models. While supervised AI-generated text detectors perform well on text generated by older LLMs, with the frequent release of new LLMs, building supervised detectors for identifying text from such new models would require new labeled training data, which is infeasible in practice. In this work, we tackle this problem and propose a domain generalization framework for the detection of AI-generated text from unseen target generators. Our proposed framework, EAGLE, leverages the labeled data that is available so far from older language models and learns features invariant across these generators, in order to detect text generated by an unknown target generator. EAGLE learns such domain-invariant features by combining the representational power of self-supervised contrastive learning with domain adversarial training. Through our experiments we demonstrate how EAGLE effectively achieves impressive performance in detecting text generated by unseen target generators, including recent state-of-the-art ones such as GPT-4 and Claude, reaching detection scores of within 4.7% of a fully supervised detector.