Toward a Flexible Framework for Linear Representation Hypothesis Using Maximum Likelihood Estimation
This work addresses a problem for researchers and practitioners in interpretability and control of LLMs, offering a more flexible framework for linear representation, though it is incremental as it builds on prior formalizations.
The paper tackles the limitation of existing linear representation frameworks that rely on single-token counterfactual pairs and cannot handle ambiguous concepts, by introducing a method called SAND that uses activation differences and maximum likelihood estimation to compute concept directions. The result is a lightweight approach that shows superior performance in activation engineering tasks like monitoring and manipulation across diverse concepts and benchmarks with LLaMA models.
Linear representation hypothesis posits that high-level concepts are encoded as linear directions in the representation spaces of LLMs. Park et al. (2024) formalize this notion by unifying multiple interpretations of linear representation, such as 1-dimensional subspace representation and interventions, using a causal inner product. However, their framework relies on single-token counterfactual pairs and cannot handle ambiguous contrasting pairs, limiting its applicability to complex or context-dependent concepts. We introduce a new notion of binary concepts as unit vectors in a canonical representation space, and utilize LLMs' (neural) activation differences along with maximum likelihood estimation (MLE) to compute concept directions (i.e., steering vectors). Our method, Sum of Activation-base Normalized Difference (SAND), formalizes the use of activation differences modeled as samples from a von Mises-Fisher (vMF) distribution, providing a principled approach to derive concept directions. We extend the applicability of Park et al. (2024) by eliminating the dependency on unembedding representations and single-token pairs. Through experiments with LLaMA models across diverse concepts and benchmarks, we demonstrate that our lightweight approach offers greater flexibility, superior performance in activation engineering tasks like monitoring and manipulation.