Base Models Look Human To AI Detectors
For educators and institutions relying on AI-text detectors, this work reveals a critical vulnerability—detectors track instruction-tuning artifacts rather than inherent machine text—and offers a simple attack, highlighting the need for more robust detector designs.
The authors find that commercial AI-text detectors (GPTZero, Pangram) judge base model outputs as human-like but flag instruction-tuned outputs as AI-generated. They propose HIP, an iterative paraphrasing pipeline that improves detector evasion while preserving semantics, and show it works across Llama-3 and Qwen-3 families (0.6B-70B).
As AI-generated text enters the real-world at scale, institutions increasingly use commercial AI-text detectors, especially in education and academic-integrity workflows. We report a surprising empirical finding about such systems: when evaluated by GPTZero and Pangram, generated text from base models is often judged overwhelmingly human, whereas text generated by their instruction-tuned counterparts is not. Building on this observation, we propose Humanization by Iterative Paraphrasing (HIP), a detector-agnostic pipeline that minimally fine-tunes a base model into a paraphraser and applies it iteratively. Compared with the baselines we test, HIP yields a stronger trade-off between semantic preservation and detector evasion on commercial detectors. Across Llama-3 and Qwen-3 families, spanning model sizes from 0.6B to 70B, HIP consistently improves detector human-likeness. Our findings suggest that current detectors are tracking artifacts of instruction tuning and local context more than any invariant notion of machine-generated text. This, in turn, calls for detector designs that model these factors more explicitly.