Efficient Computation of Shap Explanation Scores for Neural Network Classifiers via Knowledge Compilation
This addresses the efficiency problem in Explainable AI for users needing fast Shap score computations, though it is incremental as it builds on prior work on open-box classifiers.
The paper tackled the intractable computation of Shap explanation scores for neural network classifiers by transforming binary neural networks into Boolean circuits using knowledge compilation, achieving a huge performance gain as demonstrated experimentally.
The use of Shap scores has become widespread in Explainable AI. However, their computation is in general intractable, in particular when done with a black-box classifier, such as neural network. Recent research has unveiled classes of open-box Boolean Circuit classifiers for which Shap can be computed efficiently. We show how to transform binary neural networks into those circuits for efficient Shap computation.We use logic-based knowledge compilation techniques. The performance gain is huge, as we show in the light of our experiments.