A Mathematical Philosophy of Explanations in Mechanistic Interpretability -- The Strange Science Part I.i
This work addresses foundational issues in AI interpretability for researchers, but it is incremental as it builds on existing philosophical and methodological discussions without introducing new empirical results.
The paper tackles the problem of defining and justifying mechanistic interpretability as a principled approach to understanding neural networks, proposing criteria like explanatory faithfulness and a framework to distinguish it from other interpretability methods.
Mechanistic Interpretability aims to understand neural networks through causal explanations. We argue for the Explanatory View Hypothesis: that Mechanistic Interpretability research is a principled approach to understanding models because neural networks contain implicit explanations which can be extracted and understood. We hence show that Explanatory Faithfulness, an assessment of how well an explanation fits a model, is well-defined. We propose a definition of Mechanistic Interpretability (MI) as the practice of producing Model-level, Ontic, Causal-Mechanistic, and Falsifiable explanations of neural networks, allowing us to distinguish MI from other interpretability paradigms and detail MI's inherent limits. We formulate the Principle of Explanatory Optimism, a conjecture which we argue is a necessary precondition for the success of Mechanistic Interpretability.