Feature Removal Is a Unifying Principle for Model Explanation Methods
This work addresses the challenge for machine learning practitioners and researchers in choosing and comparing explanation methods, though it is incremental as it synthesizes existing approaches rather than introducing new ones.
The paper tackles the problem of understanding the relationships and selection criteria among diverse model explanation methods by identifying that many are based on the principle of feature removal, and it develops a unifying framework that categorizes 26 existing methods along three dimensions to clarify their similarities and guide usage.
Researchers have proposed a wide variety of model explanation approaches, but it remains unclear how most methods are related or when one method is preferable to another. We examine the literature and find that many methods are based on a shared principle of explaining by removing - essentially, measuring the impact of removing sets of features from a model. These methods vary in several respects, so we develop a framework for removal-based explanations that characterizes each method along three dimensions: 1) how the method removes features, 2) what model behavior the method explains, and 3) how the method summarizes each feature's influence. Our framework unifies 26 existing methods, including several of the most widely used approaches (SHAP, LIME, Meaningful Perturbations, permutation tests). Exposing the fundamental similarities between these methods empowers users to reason about which tools to use, and suggests promising directions for ongoing model explainability research.