RbX: Region-based explanations of prediction models
This work addresses the need for interpretable and reliable local explanations in machine learning, particularly for users of black-box models, but it appears incremental as it builds on existing explanation methods with a novel geometric approach.
The authors tackled the problem of generating local explanations for black-box prediction models by introducing region-based explanations (RbX), a model-agnostic method that uses a greedy algorithm to build convex polytopes approximating feature space regions where predictions are similar to a target point, and they demonstrated that RbX can more readily detect all locally relevant features than existing methods in real data and synthetic experiments.
We introduce region-based explanations (RbX), a novel, model-agnostic method to generate local explanations of scalar outputs from a black-box prediction model using only query access. RbX is based on a greedy algorithm for building a convex polytope that approximates a region of feature space where model predictions are close to the prediction at some target point. This region is fully specified by the user on the scale of the predictions, rather than on the scale of the features. The geometry of this polytope - specifically the change in each coordinate necessary to escape the polytope - quantifies the local sensitivity of the predictions to each of the features. These "escape distances" can then be standardized to rank the features by local importance. RbX is guaranteed to satisfy a "sparsity axiom," which requires that features which do not enter into the prediction model are assigned zero importance. At the same time, real data examples and synthetic experiments show how RbX can more readily detect all locally relevant features than existing methods.