A Benchmark for Interpretability Methods in Deep Neural Networks
This work addresses the need for reliable interpretability methods in machine learning, though it is incremental as it benchmarks existing approaches rather than introducing new ones.
The authors tackled the problem of evaluating the accuracy of feature importance estimates in deep neural networks, finding that many popular interpretability methods perform no better than random, with only VarGrad and SmoothGrad-Squared showing improvements.
We propose an empirical measure of the approximate accuracy of feature importance estimates in deep neural networks. Our results across several large-scale image classification datasets show that many popular interpretability methods produce estimates of feature importance that are not better than a random designation of feature importance. Only certain ensemble based approaches---VarGrad and SmoothGrad-Squared---outperform such a random assignment of importance. The manner of ensembling remains critical, we show that some approaches do no better then the underlying method but carry a far higher computational burden.