From Shapley Values to Generalized Additive Models and back
This work addresses the interpretability gap in machine learning for researchers and practitioners by offering a theoretical bridge between explanation methods, though it is incremental as it builds on existing concepts like Shapley Values and GAMs.
This paper tackles the problem of reconciling local post-hoc explanation algorithms and inherently interpretable models in explainable machine learning by establishing a correspondence between Shapley Values and Generalized Additive Models (GAMs). It introduces $n$-Shapley Values, a parametric family that covers explanations from Shapley Values to a unique functional decomposition, and shows that these recover GAMs with interaction terms up to order $n$, providing a precise characterization of Shapley Values.
In explainable machine learning, local post-hoc explanation algorithms and inherently interpretable models are often seen as competing approaches. This work offers a partial reconciliation between the two by establishing a correspondence between Shapley Values and Generalized Additive Models (GAMs). We introduce $n$-Shapley Values, a parametric family of local post-hoc explanation algorithms that explain individual predictions with interaction terms up to order $n$. By varying the parameter $n$, we obtain a sequence of explanations that covers the entire range from Shapley Values up to a uniquely determined decomposition of the function we want to explain. The relationship between $n$-Shapley Values and this decomposition offers a functionally-grounded characterization of Shapley Values, which highlights their limitations. We then show that $n$-Shapley Values, as well as the Shapley Taylor- and Faith-Shap interaction indices, recover GAMs with interaction terms up to order $n$. This implies that the original Shapely Values recover GAMs without variable interactions. Taken together, our results provide a precise characterization of Shapley Values as they are being used in explainable machine learning. They also offer a principled interpretation of partial dependence plots of Shapley Values in terms of the underlying functional decomposition. A package for the estimation of different interaction indices is available at \url{https://github.com/tml-tuebingen/nshap}.