NegBLEURT Forest: Leveraging Inconsistencies for Detecting Jailbreak Attacks
This addresses the challenge of reliably detecting jailbreak attacks that bypass safety mechanisms in LLMs, which is an incremental improvement over existing methods.
The paper tackles the problem of detecting jailbreak attacks on LLMs by analyzing semantic consistency between successful and unsuccessful responses, introducing a negation-aware scoring approach. The proposed NegBLEURT Forest method achieves top-tier performance, ranking first or second in accuracy across diverse models on a crafted dataset.
Jailbreak attacks designed to bypass safety mechanisms pose a serious threat by prompting LLMs to generate harmful or inappropriate content, despite alignment with ethical guidelines. Crafting universal filtering rules remains difficult due to their inherent dependence on specific contexts. To address these challenges without relying on threshold calibration or model fine-tuning, this work introduces a semantic consistency analysis between successful and unsuccessful responses, demonstrating that a negation-aware scoring approach captures meaningful patterns. Building on this insight, a novel detection framework called NegBLEURT Forest is proposed to evaluate the degree of alignment between outputs elicited by adversarial prompts and expected safe behaviors. It identifies anomalous responses using the Isolation Forest algorithm, enabling reliable jailbreak detection. Experimental results show that the proposed method consistently achieves top-tier performance, ranking first or second in accuracy across diverse models using the crafted dataset, while competing approaches exhibit notable sensitivity to model and data variations.