Trace Is In Sentences: Unbiased Lightweight ChatGPT-Generated Text Detector
This addresses misuse concerns for ChatGPT users and platforms by improving detection robustness, though it is incremental as it builds on existing sentence-level and bias mitigation techniques.
The paper tackles the problem of detecting AI-generated text, including paraphrased versions, by proposing a lightweight framework that classifies texts based on invariant internal sentence structures, achieving validated effectiveness on curated datasets.
The widespread adoption of ChatGPT has raised concerns about its misuse, highlighting the need for robust detection of AI-generated text. Current word-level detectors are vulnerable to paraphrasing or simple prompts (PSP), suffer from biases induced by ChatGPT's word-level patterns (CWP) and training data content, degrade on modified text, and often require large models or online LLM interaction. To tackle these issues, we introduce a novel task to detect both original and PSP-modified AI-generated texts, and propose a lightweight framework that classifies texts based on their internal structure, which remains invariant under word-level changes. Our approach encodes sentence embeddings from pre-trained language models and models their relationships via attention. We employ contrastive learning to mitigate embedding biases from autoregressive generation and incorporate a causal graph with counterfactual methods to isolate structural features from topic-related biases. Experiments on two curated datasets, including abstract comparisons and revised life FAQs, validate the effectiveness of our method.