Probabilistic Concept-Aware Steering for Trustworthy LLM Inference
Addresses the lack of fine-grained, interpretable control in LLM inference-time steering for safety-critical applications.
Existing steering vector methods for LLMs suffer from representation-incoherent behaviors due to binary evaluation and discrete clustering metrics. The proposed PCS framework achieves controllable, safety-oriented semantic bias while preserving task competence through probabilistic calibration.
Steering vectors (SVs), an inference-time intervention technique for large language models (LLMs), guide the generation process by adding a concept-specific direction vector to intermediate activations during inference. However, existing SV methods frequently yield representation-incoherent behaviors that undermine interpretability and fine-grained control, largely because prior work has focused on binary positive-negative steering evaluation while employing discrete clustering metrics that fail to capture the continuous spectrum of semantic alignment. In this work, we present the Probabilistic Concept-Aware Steering (PCS) framework for LLM inference. PCS preserves original task competence while providing controllable, safety-oriented semantic bias through concept-driven steering-vector retrieval and probabilistic strength calibration.