The Linear Centroids Hypothesis: How Deep Network Features Represent Data
For researchers in deep network interpretability, the LCH provides a new paradigm that addresses limitations of the Linear Representation Hypothesis, offering sparser and more effective feature extraction.
The paper introduces the Linear Centroids Hypothesis (LCH) as a new framework for identifying features in deep networks, showing that applying sparse autoencoders to centroids yields sparser feature dictionaries for DINO vision transformers with better downstream performance, and enables circuit identification in GPT2-Large.
Identifying and understanding the features that a deep network (DN) extracts from its inputs to produce its outputs is a focal point of interpretability research. The Linear Representation Hypothesis (LRH) identifies features in terms of the linear directions formed by the inputs in a DN's latent space. However, the LRH is limited as it abstracts away from individual components (e.g., neurons and layers), is susceptible to identifying spurious features, and cannot be applied across sub-components (e.g., multiple layers). In this paper, we introduce the Linear Centroids Hypothesis (LCH) as a new framework for identifying the features of a DN. The LCH posits that features correspond to linear directions of centroids, which are vector summarizations of the functional behavior of a DN in a local region of its input space. Interpretability studies under the LCH can leverage existing LRH tools, such as sparse autoencoders, by applying them to the DN's centroids rather than to its latent activations. We demonstrate that doing so yields sparser feature dictionaries for DINO vision transformers, which also perform better on downstream tasks. The LCH also inspires novel approaches to interpretability; for example, LCH can readily identify circuits in GPT2-Large. For code to study the LCH https://github.com/ThomasWalker1/LinearCentroidsHypothesis .