InterpBench: Semi-Synthetic Transformers for Evaluating Mechanistic Interpretability Techniques
This provides a benchmark for evaluating interpretability techniques in AI, but it is incremental as it builds on existing tools like Tracr and IIT.
The authors tackled the problem of validating mechanistic interpretability methods by introducing InterpBench, a collection of semi-synthetic transformers with known circuits, and found that their Strict IIT method maintains original circuits while being more realistic and can train larger circuits like IOI.
Mechanistic interpretability methods aim to identify the algorithm a neural network implements, but it is difficult to validate such methods when the true algorithm is unknown. This work presents InterpBench, a collection of semi-synthetic yet realistic transformers with known circuits for evaluating these techniques. We train simple neural networks using a stricter version of Interchange Intervention Training (IIT) which we call Strict IIT (SIIT). Like the original, SIIT trains neural networks by aligning their internal computation with a desired high-level causal model, but it also prevents non-circuit nodes from affecting the model's output. We evaluate SIIT on sparse transformers produced by the Tracr tool and find that SIIT models maintain Tracr's original circuit while being more realistic. SIIT can also train transformers with larger circuits, like Indirect Object Identification (IOI). Finally, we use our benchmark to evaluate existing circuit discovery techniques.