PredDiff: Explanations and Interactions from Conditional Expectations
This work addresses the need for reliable and interpretable explanations in machine learning, particularly for scientific applications, by providing a novel interaction measure that enhances model understanding, though it builds incrementally on existing attribution methods.
The authors tackled the problem of explaining black-box models by introducing PredDiff, a model-agnostic attribution method based on conditional expectations, and extended it with a new measure for interaction effects between features, resulting in a theoretically sound and numerically efficient approach.
PredDiff is a model-agnostic, local attribution method that is firmly rooted in probability theory. Its simple intuition is to measure prediction changes while marginalizing features. In this work, we clarify properties of PredDiff and its close connection to Shapley values. We stress important differences between classification and regression, which require a specific treatment within both formalisms. We extend PredDiff by introducing a new, well-founded measure for interaction effects between arbitrary feature subsets. The study of interaction effects represents an inevitable step towards a comprehensive understanding of black-box models and is particularly important for science applications. Equipped with our novel interaction measure, PredDiff is a promising model-agnostic approach for obtaining reliable, numerically inexpensive and theoretically sound attributions.