Efficient nonparametric statistical inference on population feature importance using Shapley values
This work addresses the need for computationally efficient and statistically valid inference on variable importance in prediction tasks, which is crucial for understanding data mechanisms and guiding experimental design, though it is incremental as it builds on existing Shapley value methods.
The paper tackles the problem of performing valid statistical inference on population-level variable importance using Shapley values, which is computationally expensive due to exponential scaling with variables. It proposes an efficient estimator based on random sampling of feature subsets, achieving asymptotically optimal convergence and enabling confidence intervals and hypothesis tests with good finite-sample performance in simulations and real-world mortality prediction tasks.
The true population-level importance of a variable in a prediction task provides useful knowledge about the underlying data-generating mechanism and can help in deciding which measurements to collect in subsequent experiments. Valid statistical inference on this importance is a key component in understanding the population of interest. We present a computationally efficient procedure for estimating and obtaining valid statistical inference on the Shapley Population Variable Importance Measure (SPVIM). Although the computational complexity of the true SPVIM scales exponentially with the number of variables, we propose an estimator based on randomly sampling only $Θ(n)$ feature subsets given $n$ observations. We prove that our estimator converges at an asymptotically optimal rate. Moreover, by deriving the asymptotic distribution of our estimator, we construct valid confidence intervals and hypothesis tests. Our procedure has good finite-sample performance in simulations, and for an in-hospital mortality prediction task produces similar variable importance estimates when different machine learning algorithms are applied.