Axiomatic Explainer Globalness via Optimal Transport
This work addresses the problem of selecting explainability methods for practitioners by providing a quantitative metric, though it is incremental as it builds on existing explainability evaluation frameworks.
The paper tackles the challenge of evaluating and comparing explainability methods by introducing a complexity measure called globalness, which quantifies the diversity of explanations produced by feature attribution and selection methods, and validates its utility across image, tabular, and synthetic datasets to improve explainer selection.
Explainability methods are often challenging to evaluate and compare. With a multitude of explainers available, practitioners must often compare and select explainers based on quantitative evaluation metrics. One particular differentiator between explainers is the diversity of explanations for a given dataset; i.e. whether all explanations are identical, unique and uniformly distributed, or somewhere between these two extremes. In this work, we define a complexity measure for explainers, globalness, which enables deeper understanding of the distribution of explanations produced by feature attribution and feature selection methods for a given dataset. We establish the axiomatic properties that any such measure should possess and prove that our proposed measure, Wasserstein Globalness, meets these criteria. We validate the utility of Wasserstein Globalness using image, tabular, and synthetic datasets, empirically showing that it both facilitates meaningful comparison between explainers and improves the selection process for explainability methods.