PhantomHunter: Detecting Unseen Privately-Tuned LLM-Generated Text via Family-Aware Learning
This addresses a critical issue for detecting misinformation and academic misconduct in real-world applications, though it is incremental as it builds on existing detection methods.
The paper tackles the problem of detecting text generated by unseen, privately-tuned large language models (LLMs), which existing detectors struggle with, and proposes PhantomHunter, achieving F1 scores over 96% in experiments.
With the popularity of large language models (LLMs), undesirable societal problems like misinformation production and academic misconduct have been more severe, making LLM-generated text detection now of unprecedented importance. Although existing methods have made remarkable progress, a new challenge posed by text from privately tuned LLMs remains underexplored. Users could easily possess private LLMs by fine-tuning an open-source one with private corpora, resulting in a significant performance drop of existing detectors in practice. To address this issue, we propose PhantomHunter, an LLM-generated text detector specialized for detecting text from unseen, privately-tuned LLMs. Its family-aware learning framework captures family-level traits shared across the base models and their derivatives, instead of memorizing individual characteristics. Experiments on data from LLaMA, Gemma, and Mistral families show its superiority over 7 baselines and 3 industrial services, with F1 scores of over 96%.