Mechanistic Indicators of Steering Effectiveness in Large Language Models
This addresses the problem of understanding when steering succeeds or fails in LLMs for researchers and practitioners, but it is incremental as it builds on existing methods with new diagnostic measures.
The study investigated whether internal model signals, specifically entropy-derived Normalized Branching Factor and KL divergence, can diagnose the reliability of activation-based steering in LLMs, showing these signals provide predictive power for identifying successful steering and estimating failure probability.
Activation-based steering enables Large Language Models (LLMs) to exhibit targeted behaviors by intervening on intermediate activations without retraining. Despite its widespread use, the mechanistic factors that govern when steering succeeds or fails remain poorly understood, as prior work has relied primarily on black-box outputs or LLM-based judges. In this study, we investigate whether the reliability of steering can be diagnosed using internal model signals. We focus on two information-theoretic measures: the entropy-derived Normalized Branching Factor (NBF), and the Kullback-Leibler (KL) divergence between steered activations and targeted concepts in the vocabulary space. We hypothesize that effective steering corresponds to structured entropy preservation and coherent KL alignment across decoding steps. Building on a reliability study demonstrating high inter-judge agreement between two architecturally distinct LLMs, we use LLM-generated annotations as ground truth and show that these mechanistic signals provide meaningful predictive power for identifying successful steering and estimating failure probability. We further introduce a stronger evaluation baseline for Contrastive Activation Addition (CAA) and Sparse Autoencoder-based steering, the two most widely adopted activation-steering methods.