Can We Faithfully Represent Masked States to Compute Shapley Values on a DNN?
This addresses the issue of accurate attribution in deep learning for researchers and practitioners, but it is incremental as it builds on existing causal pattern frameworks.
The paper tackles the problem of whether baseline values faithfully represent the absence of input variables when computing Shapley values for DNN attributions, and it proposes a method to learn optimal baseline values, showing effectiveness in experiments.
Masking some input variables of a deep neural network (DNN) and computing output changes on the masked input sample represent a typical way to compute attributions of input variables in the sample. People usually mask an input variable using its baseline value. However, there is no theory to examine whether baseline value faithfully represents the absence of an input variable, \emph{i.e.,} removing all signals from the input variable. Fortunately, recent studies show that the inference score of a DNN can be strictly disentangled into a set of causal patterns (or concepts) encoded by the DNN. Therefore, we propose to use causal patterns to examine the faithfulness of baseline values. More crucially, it is proven that causal patterns can be explained as the elementary rationale of the Shapley value. Furthermore, we propose a method to learn optimal baseline values, and experimental results have demonstrated its effectiveness.