Feature-Augmented Transformers for Robust AI-Text Detection Across Domains and Generators
For practitioners deploying AI-text detectors in real-world settings, this work provides a more robust and realistic evaluation protocol and a method that improves cross-domain/generator transfer.
The paper addresses the brittleness of AI-text detectors under distribution shift across domains and generators. By augmenting a DeBERTa-v3-base transformer with attention-based linguistic feature fusion, they achieve 85.9% balanced accuracy on the M4 benchmark, outperforming zero-shot baselines by up to +7.22 points under a fixed-threshold protocol.
AI-generated text is nowadays produced at scale across domains and heterogeneous generation pipelines, making robustness to distribution shift a central requirement for supervised binary detectors. We train transformer-based detectors on HC3 PLUS and calibrate a single decision threshold by maximising balanced accuracy on held-out validation; this threshold is then kept fixed for all downstream test distributions, revealing domain- and generator-dependent error asymmetries under shift. We evaluate in-domain on HC3 PLUS, under cross-dataset transfer to the multi-domain, multi-generator M4 benchmark, and on the external AI-Text-Detection-Pile. Although base models achieve near-ceiling in-domain performance (up to 99.5% balanced accuracy), performance under shift is brittle and strongly model-dependent. Feature augmentation via attention-based linguistic feature fusion improves transfer, with our best model (DeBERTa-v3-base+FeatAttn) achieving 85.9% balanced accuracy on M4. Multi-seed experiments confirm high stability. Under the same fixed-threshold protocol, our model outperforms strong zero-shot baselines by up to +7.22 points. Category-level ablations further show that readability and vocabulary features contribute most to robustness under shift. Overall, these results demonstrate that feature augmentation and a modern DeBERTa backbone significantly outperform earlier BERT/RoBERTa models, while the fixed-threshold protocol provides a more realistic and informative assessment of practical detector robustness.