Propositional Interpretability in Artificial Intelligence
This work addresses the interpretability problem for AI researchers and philosophers, but it is incremental as it builds on existing concepts without introducing a novel solution.
The paper argues for the importance of propositional interpretability in AI, which involves explaining AI systems in terms of propositional attitudes like belief or desire, and identifies thought logging as a key challenge, assessing current methods such as probing and chain of thought without presenting new experimental results.
Mechanistic interpretability is the program of explaining what AI systems are doing in terms of their internal mechanisms. I analyze some aspects of the program, along with setting out some concrete challenges and assessing progress to date. I argue for the importance of propositional interpretability, which involves interpreting a system's mechanisms and behavior in terms of propositional attitudes: attitudes (such as belief, desire, or subjective probability) to propositions (e.g. the proposition that it is hot outside). Propositional attitudes are the central way that we interpret and explain human beings and they are likely to be central in AI too. A central challenge is what I call thought logging: creating systems that log all of the relevant propositional attitudes in an AI system over time. I examine currently popular methods of interpretability (such as probing, sparse auto-encoders, and chain of thought methods) as well as philosophical methods of interpretation (including those grounded in psychosemantics) to assess their strengths and weaknesses as methods of propositional interpretability.