Prefix Probing: Lightweight Harmful Content Detection for Large Language Models
This addresses the problem of efficient safety filtering for LLM users in real-world applications, though it is incremental as it builds on existing prefix-based techniques.
The paper tackles the trade-off between accuracy, latency, and cost in harmful content detection for large language models by introducing Prefix Probing, a method that uses prefix comparisons and caching to achieve near first-token latency. It demonstrates detection effectiveness comparable to external safety models with minimal computational cost and no extra deployment.
Large language models often face a three-way trade-off among detection accuracy, inference latency, and deployment cost when used in real-world safety-sensitive applications. This paper introduces Prefix Probing, a black-box harmful content detection method that compares the conditional log-probabilities of "agreement/execution" versus "refusal/safety" opening prefixes and leverages prefix caching to reduce detection overhead to near first-token latency. During inference, the method requires only a single log-probability computation over the probe prefixes to produce a harmfulness score and apply a threshold, without invoking any additional models or multi-stage inference. To further enhance the discriminative power of the prefixes, we design an efficient prefix construction algorithm that automatically discovers highly informative prefixes, substantially improving detection performance. Extensive experiments demonstrate that Prefix Probing achieves detection effectiveness comparable to mainstream external safety models while incurring only minimal computational cost and requiring no extra model deployment, highlighting its strong practicality and efficiency.