Explaining a Series of Models by Propagating Shapley Values
This addresses the need for efficient and applicable explanations in domains like finance where distributed models are common and explanations are mandated, representing a novel method for a known bottleneck.
The authors tackled the problem of explaining distributed series of models, which is costly or infeasible with existing methods, by introducing DeepSHAP, a tractable method based on Shapley values that provides equally salient explanations an order of magnitude faster than current techniques.
Local feature attribution methods are increasingly used to explain complex machine learning models. However, current methods are limited because they are extremely expensive to compute or are not capable of explaining a distributed series of models where each model is owned by a separate institution. The latter is particularly important because it often arises in finance where explanations are mandated. Here, we present DeepSHAP, a tractable method to propagate local feature attributions through complex series of models based on a connection to the Shapley value. We evaluate DeepSHAP across biological, health, and financial datasets to show that it provides equally salient explanations an order of magnitude faster than existing model-agnostic attribution techniques and demonstrate its use in an important distributed series of models setting.