SCANS: Mitigating the Exaggerated Safety for LLMs via Safety-Conscious Activation Steering
This addresses the exaggerated safety problem in LLMs, which limits their helpfulness for users, representing an incremental improvement in balancing safety and utility.
The paper tackles the problem of safety-aligned large language models (LLMs) rejecting benign queries due to exaggerated safety concerns, proposing the SCANS method to mitigate this issue while maintaining safety and model capability, achieving state-of-the-art performance on benchmarks like XSTest and OKTest.
Safety alignment is indispensable for Large Language Models (LLMs) to defend threats from malicious instructions. However, recent researches reveal safety-aligned LLMs prone to reject benign queries due to the exaggerated safety issue, limiting their helpfulness. In this paper, we propose a Safety-Conscious Activation Steering (SCANS) method to mitigate the exaggerated safety concerns in aligned LLMs. First, SCANS extracts the refusal steering vectors within the activation space and utilizes vocabulary projection to anchor some specific safety-critical layers which influence model refusal behavior. Second, by tracking the hidden state transition, SCANS identifies the steering direction and steers the model behavior accordingly, achieving a balance between exaggerated safety and adequate safety. Experiments show that SCANS achieves new state-of-the-art performance on XSTest and OKTest benchmarks, without impairing their defense capability against harmful queries and maintaining almost unchanged model capability.