The Logical Implication Steering Method for Conditional Interventions on Transformer Generation
This work introduces a method for transparent and interpretable conditional interventions in transformer models, which is incremental as it builds on existing mechanistic interpretability findings.
The paper tackles the problem of steering transformer model generation behavior by integrating logical implication, enabling interpretable adjustments that induce chosen generation responses to given concepts.
The field of mechanistic interpretability in pre-trained transformer models has demonstrated substantial evidence supporting the ''linear representation hypothesis'', which is the idea that high level concepts are encoded as vectors in the space of activations of a model. Studies also show that model generation behavior can be steered toward a given concept by adding the concept's vector to the corresponding activations. We show how to leverage these properties to build a form of logical implication into models, enabling transparent and interpretable adjustments that induce a chosen generation behavior in response to the presence of any given concept. Our method, Logical Implication Model Steering (LIMS), unlocks new hand engineered reasoning capabilities by integrating neuro-symbolic logic into pre-trained transformer models.