Identifying Linear Relational Concepts in Large Language Models
This provides a method for better understanding and controlling concept representations in large language models, with potential applications in interpretability and model editing.
The paper tackles the problem of identifying human-interpretable concept directions in the latent space of transformer language models by introducing linear relational concepts (LRC), which outperforms standard probing classifiers in concept classification and causal model output changes.
Transformer language models (LMs) have been shown to represent concepts as directions in the latent space of hidden activations. However, for any human-interpretable concept, how can we find its direction in the latent space? We present a technique called linear relational concepts (LRC) for finding concept directions corresponding to human-interpretable concepts by first modeling the relation between subject and object as a linear relational embedding (LRE). We find that inverting the LRE and using earlier object layers results in a powerful technique for finding concept directions that outperforms standard black-box probing classifiers. We evaluate LRCs on their performance as concept classifiers as well as their ability to causally change model output.