HyperDAS: Towards Automating Mechanistic Interpretability with Hypernetworks
This addresses the problem of automating interpretability for researchers, though it is incremental as it builds on DAS with a more efficient method.
The paper tackles the computational inefficiency of brute-force search in distributed alignment search (DAS) for mechanistic interpretability by introducing HyperDAS, a transformer-based hypernetwork that automatically locates and constructs concept features in neural network hidden states, achieving state-of-the-art performance on the RAVEL benchmark with Llama3-8B.
Mechanistic interpretability has made great strides in identifying neural network features (e.g., directions in hidden activation space) that mediate concepts(e.g., the birth year of a person) and enable predictable manipulation. Distributed alignment search (DAS) leverages supervision from counterfactual data to learn concept features within hidden states, but DAS assumes we can afford to conduct a brute force search over potential feature locations. To address this, we present HyperDAS, a transformer-based hypernetwork architecture that (1) automatically locates the token-positions of the residual stream that a concept is realized in and (2) constructs features of those residual stream vectors for the concept. In experiments with Llama3-8B, HyperDAS achieves state-of-the-art performance on the RAVEL benchmark for disentangling concepts in hidden states. In addition, we review the design decisions we made to mitigate the concern that HyperDAS (like all powerful interpretabilty methods) might inject new information into the target model rather than faithfully interpreting it.