An Imprecise SHAP as a Tool for Explaining the Class Probability Distributions under Limited Training Data
This work addresses the challenge of providing reliable explanations for AI models in data-scarce scenarios, representing an incremental improvement to existing SHAP methods.
The paper tackles the problem of explaining machine learning predictions when class probability distributions are imprecise due to limited training data, proposing an imprecise SHAP method that modifies the original SHAP to handle sets of distributions and demonstrates it with numerical examples.
One of the most popular methods of the machine learning prediction explanation is the SHapley Additive exPlanations method (SHAP). An imprecise SHAP as a modification of the original SHAP is proposed for cases when the class probability distributions are imprecise and represented by sets of distributions. The first idea behind the imprecise SHAP is a new approach for computing the marginal contribution of a feature, which fulfils the important efficiency property of Shapley values. The second idea is an attempt to consider a general approach to calculating and reducing interval-valued Shapley values, which is similar to the idea of reachable probability intervals in the imprecise probability theory. A simple special implementation of the general approach in the form of linear optimization problems is proposed, which is based on using the Kolmogorov-Smirnov distance and imprecise contamination models. Numerical examples with synthetic and real data illustrate the imprecise SHAP.