Evaluating and Aggregating Feature-based Model Explanations
This work addresses the need for practitioners to choose among explanation functions, though it is incremental as it builds on existing explanation methods.
The paper tackles the problem of evaluating feature-based model explanations by proposing quantitative criteria (low sensitivity, high faithfulness, low complexity) and develops a framework for aggregating explanation functions, resulting in a new aggregate Shapley value function that minimizes sensitivity.
A feature-based model explanation denotes how much each input feature contributes to a model's output for a given data point. As the number of proposed explanation functions grows, we lack quantitative evaluation criteria to help practitioners know when to use which explanation function. This paper proposes quantitative evaluation criteria for feature-based explanations: low sensitivity, high faithfulness, and low complexity. We devise a framework for aggregating explanation functions. We develop a procedure for learning an aggregate explanation function with lower complexity and then derive a new aggregate Shapley value explanation function that minimizes sensitivity.