Beyond "I cannot fulfill this request": Alleviating Rigid Rejection in LLMs via Label Enhancement
For LLM safety alignment, LANCE addresses the problem of rigid rejection that undermines interaction naturalness, offering a more flexible refusal mechanism.
LANCE alleviates rigid rejection in LLMs by using variational inference to predict fine-grained rejection distributions, enabling safe yet natural responses. It significantly outperforms baselines in helpfulness and naturalness while maintaining security.
Large Language Models (LLMs) rely on safety alignment to obey safe requests while refusing harmful ones. However, traditional refusal mechanisms often lead to "rigid rejection," where a general template (e.g., "I cannot fulfill this request") indiscriminately triggers refusals and severely undermines the naturalness of interactions between humans and LLMs. To address this issue, LANCE is proposed in this paper to ensure safe yet flexible and natural responses via label enhancement. Specifically, LANCE employs variational inference to perform label enhancement, predicting a continuous distribution across multiple rejection categories. These fine-grained rejection distributions provide multi-way textual gradients for a refinement model to neutralize the hazardous elements in the prompt, so that the LLMs could generate safe responses that avoid rigid rejections while preserving the naturalness of interactions. Experiments demonstrate that LANCE significantly alleviates the rigid rejection problem while maintaining high security standards, significantly outperforming existing baseline models in terms of helpfulness and naturalness of responses.