Efficient and Accurate Explanation Estimation with Distribution Compression
This addresses the problem of slow and inaccurate explanation estimation for users of post-hoc explainability methods, offering a plug-in solution that is incremental but provides significant speed and accuracy gains.
The paper tackles the computational inefficiency of approximating machine learning explanations by introducing Compress Then Explain (CTE), a new paradigm that uses distribution compression to reduce approximation error and achieve on-par results 2-3x faster with fewer model evaluations.
We discover a theoretical connection between explanation estimation and distribution compression that significantly improves the approximation of feature attributions, importance, and effects. While the exact computation of various machine learning explanations requires numerous model inferences and becomes impractical, the computational cost of approximation increases with an ever-increasing size of data and model parameters. We show that the standard i.i.d. sampling used in a broad spectrum of algorithms for post-hoc explanation leads to an approximation error worthy of improvement. To this end, we introduce Compress Then Explain (CTE), a new paradigm of sample-efficient explainability. It relies on distribution compression through kernel thinning to obtain a data sample that best approximates its marginal distribution. CTE significantly improves the accuracy and stability of explanation estimation with negligible computational overhead. It often achieves an on-par explanation approximation error 2-3x faster by using fewer samples, i.e. requiring 2-3x fewer model evaluations. CTE is a simple, yet powerful, plug-in for any explanation method that now relies on i.i.d. sampling.