The Shape of Explanations: A Topological Account of Rule-Based Explanations in Machine Learning
This provides a theoretical basis for evaluating explanation methods, which is incremental as it builds on existing methods like Anchors and LORE without introducing new practical techniques.
The paper tackles the lack of a general theoretical foundation for rule-based explanation methods in machine learning by introducing a topological framework to characterize explainability based on classifier definability relative to explanation schemes, showing that the optimal scheme depends on user knowledge of the domain and probability measure.
Rule-based explanations provide simple reasons explaining the behavior of machine learning classifiers at given points in the feature space. Several recent methods (Anchors, LORE, etc.) purport to generate rule-based explanations for arbitrary or black-box classifiers. But what makes these methods work in general? We introduce a topological framework for rule-based explanation methods and provide a characterization of explainability in terms of the definability of a classifier relative to an explanation scheme. We employ this framework to consider various explanation schemes and argue that the preferred scheme depends on how much the user knows about the domain and the probability measure over the feature space.