Segment-Level Coherence for Robust Harmful Intent Probing in LLMs
For safety-critical LLM monitoring, this work provides a robust detection method that reduces false alarms from isolated sensitive terms, with strong empirical gains.
The paper addresses the problem of false alarms in streaming harmful intent probes for LLMs, particularly in CBRN domains, by introducing a segment-level coherence objective that aggregates evidence across tokens. At a 1% false-positive rate, this method improves true-positive rate by 35.55% relative to strong baselines and achieves over 98.85% AUROC on obfuscated attacks.
Large Language Models (LLMs) are increasingly exposed to adaptive jailbreaking, particularly in high-stakes Chemical, Biological, Radiological, and Nuclear (CBRN) domains. Although streaming probes enable real-time monitoring, they still make systematic errors. We identify a core issue: existing methods often rely on a few high-scoring tokens, leading to false alarms when sensitive CBRN terms appear in benign contexts. To address this, we introduce a streaming probing objective that requires multiple evidence tokens to consistently support a prediction, rather than relying on isolated spikes. This encourages more robust detection based on aggregated signals instead of single-token cues. At a fixed 1% false-positive rate, our method improves the true-positive rate by 35.55% relative to strong streaming baselines. We further observe substantial gains in AUROC, even when starting from near-saturated baseline performance (AUROC = 97.40%). We also show that probing Attention or MLP activations consistently outperforms residual-stream features. Finally, even when adversarial fine-tuning enables novel character-level ciphers, harmful intent remains detectable: probes developed for the base LLMs can be applied ``plug-and-play'' to these obfuscated attacks, achieving an AUROC of over 98.85%.