Shapley Flow: A Graph-based Approach to Interpreting Model Predictions
This work addresses the problem of interpreting machine learning models for researchers and practitioners by offering a more comprehensive graph-based method, but it appears incremental as it builds on existing Shapley value and causal graph approaches.
The authors tackled the problem of feature importance estimation by proposing Shapley Flow, a novel approach that assigns credit to edges in a causal graph instead of nodes, addressing dependencies among features and enabling a graph-based view of importance flow. They demonstrated its benefits for interpreting model predictions and reasoning about interventions, though no concrete numbers were provided.
Many existing approaches for estimating feature importance are problematic because they ignore or hide dependencies among features. A causal graph, which encodes the relationships among input variables, can aid in assigning feature importance. However, current approaches that assign credit to nodes in the causal graph fail to explain the entire graph. In light of these limitations, we propose Shapley Flow, a novel approach to interpreting machine learning models. It considers the entire causal graph, and assigns credit to \textit{edges} instead of treating nodes as the fundamental unit of credit assignment. Shapley Flow is the unique solution to a generalization of the Shapley value axioms to directed acyclic graphs. We demonstrate the benefit of using Shapley Flow to reason about the impact of a model's input on its output. In addition to maintaining insights from existing approaches, Shapley Flow extends the flat, set-based, view prevalent in game theory based explanation methods to a deeper, \textit{graph-based}, view. This graph-based view enables users to understand the flow of importance through a system, and reason about potential interventions.