Evaluating Explanations: An Explanatory Virtues Framework for Mechanistic Interpretability -- The Strange Science Part I.ii
This work addresses a foundational bottleneck in AI interpretability, offering a systematic way to evaluate explanations, which is incremental but could enhance monitoring and steering of AI systems.
The paper tackles the lack of a universal evaluation approach for explanations in Mechanistic Interpretability by introducing an Explanatory Virtues Framework based on Philosophy of Science perspectives, finding that Compact Proofs are promising and suggesting research directions like defining simplicity and deriving universal principles.
Mechanistic Interpretability (MI) aims to understand neural networks through causal explanations. Though MI has many explanation-generating methods, progress has been limited by the lack of a universal approach to evaluating explanations. Here we analyse the fundamental question "What makes a good explanation?" We introduce a pluralist Explanatory Virtues Framework drawing on four perspectives from the Philosophy of Science - the Bayesian, Kuhnian, Deutschian, and Nomological - to systematically evaluate and improve explanations in MI. We find that Compact Proofs consider many explanatory virtues and are hence a promising approach. Fruitful research directions implied by our framework include (1) clearly defining explanatory simplicity, (2) focusing on unifying explanations and (3) deriving universal principles for neural networks. Improved MI methods enhance our ability to monitor, predict, and steer AI systems.