LLM Encoder vs. Decoder: Robust Detection of Chinese AI-Generated Text with LoRA
This addresses the challenge of robust AI-generated text detection in Chinese for applications like content moderation, though it is incremental as it applies existing methods (LoRA fine-tuning) to a specific language domain.
The study tackled the problem of detecting AI-generated text in Chinese by systematically comparing encoder-based Transformers, a decoder-only LLM fine-tuned with LoRA, and FastText on a public dataset. The LoRA-adapted Qwen2.5-7B achieved 95.94% test accuracy, significantly outperforming encoder models (76.3-79.3%) and FastText (83.5%), demonstrating superior generalization.
The rapid growth of large language models (LLMs) has heightened the demand for accurate detection of AI-generated text, particularly in languages like Chinese, where subtle linguistic nuances pose significant challenges to current methods. In this study, we conduct a systematic comparison of encoder-based Transformers (Chinese BERT-large and RoBERTa-wwm-ext-large), a decoder-only LLM (Alibaba's Qwen2.5-7B/DeepSeek-R1-Distill-Qwen-7B fine-tuned via Low-Rank Adaptation, LoRA), and a FastText baseline using the publicly available dataset from the NLPCC 2025 Chinese AI-Generated Text Detection Task. Encoder models were fine-tuned using a novel prompt-based masked language modeling approach, while Qwen2.5-7B was adapted for classification with an instruction-format input and a lightweight classification head trained via LoRA. Experiments reveal that although encoder models nearly memorize training data, they suffer significant performance degradation under distribution shifts (RoBERTa: 76.3% test accuracy; BERT: 79.3%). FastText demonstrates surprising lexical robustness (83.5% accuracy) yet lacks deeper semantic understanding. In contrast, the LoRA-adapted Qwen2.5-7B achieves 95.94% test accuracy with balanced precision-recall metrics, indicating superior generalization and resilience to dataset-specific artifacts. These findings underscore the efficacy of decoder-based LLMs with parameter-efficient fine-tuning for robust Chinese AI-generated text detection. Future work will explore next-generation Qwen3 models, distilled variants, and ensemble strategies to enhance cross-domain robustness further.