Evaluating Local Explanations using White-box Models
This work addresses the challenge of costly and subjective human evaluations for explanation techniques in machine learning, though it is incremental as it builds on existing methods for specific model types.
The paper tackles the problem of evaluating local explanation techniques by proposing to use the log odds ratio (LOR) of prediction functions, which naturally decomposes into additive feature importance scores for models like logistic regression and naive Bayes, and demonstrates benchmarking of techniques based on similarity to LOR scores in experiments.
Evaluating explanation techniques using human subjects is costly, time-consuming and can lead to subjectivity in the assessments. To evaluate the accuracy of local explanations, we require access to the true feature importance scores for a given instance. However, the prediction function of a model usually does not decompose into linear additive terms that indicate how much a feature contributes to the output. In this work, we suggest to instead focus on the log odds ratio (LOR) of the prediction function, which naturally decomposes into additive terms for logistic regression and naive Bayes. We demonstrate how we can benchmark different explanation techniques in terms of their similarity to the LOR scores based on our proposed approach. In the experiments, we compare prominent local explanation techniques and find that the performance of the techniques can depend on the underlying model, the dataset, which data point is explained, the normalization of the data and the similarity metric.