Explaining Models by Propagating Shapley Values of Local Components
This work addresses the need for explainability in healthcare AI to improve patient welfare, but it is incremental as it builds upon existing methods like DeepLIFT.
The paper tackles the problem of explaining complex models like neural networks and mixed-model stacks in healthcare by introducing DeepSHAP, a framework for propagating Shapley values layer-wise, building on DeepLIFT, and it theoretically justifies attributions with respect to a background distribution.
In healthcare, making the best possible predictions with complex models (e.g., neural networks, ensembles/stacks of different models) can impact patient welfare. In order to make these complex models explainable, we present DeepSHAP for mixed model types, a framework for layer wise propagation of Shapley values that builds upon DeepLIFT (an existing approach for explaining neural networks). We show that in addition to being able to explain neural networks, this new framework naturally enables attributions for stacks of mixed models (e.g., neural network feature extractor into a tree model) as well as attributions of the loss. Finally, we theoretically justify a method for obtaining attributions with respect to a background distribution (under a Shapley value framework).