Pangram 4 Technical Report
For AI-text detection practitioners, Pangram 4 provides a highly accurate and robust model with improved fine-grained detection capabilities.
Pangram 4 achieves state-of-the-art AI-text detection with AUROC 0.9916, false positive rate 0.0041%, and false negative rate 0.3396%, outperforming Pangram 3 in accuracy, out-of-distribution generalization, adversarial robustness, and detection of mixed AI-human text.
We present Pangram 4, the latest deep-learning-based AI-text classification model from Pangram Labs. We achieve an AUROC of 0.9916 with a false positive rate of 0.0041% and a false negative rate of 0.3396%. In addition to its increased overall accuracy compared with Pangram 3, Pangram 4 exhibits superior out-of-distribution generalization and robustness to adversarial attacks. Another novel contribution of Pangram 4 is its improved ability to distinguish fine-grained edits and mixed AI-human co-authored text. We demonstrate improvements to both boundary detection tasks and the detection of interleaved AI assistance. Finally, we report metrics on standard AI detection benchmarks showing that Pangram 4 achieves state-of-the-art performance on the AI text detection task across a wide variety of settings and domains.