Measuring the Effect of Training Data on Deep Learning Predictions via Randomized Experiments
This work addresses the need for interpretability in deep learning by providing a principled way to measure data point effects, which is incremental as it builds on existing Shapley value concepts with efficiency gains.
The paper tackles the problem of quantifying individual training data contributions to deep learning model predictions by introducing a new algorithm that estimates the Average Marginal Effect (AME) using randomized experiments and LASSO regression. The result is an efficient estimator requiring only O(k log N) submodel trainings under sparsity assumptions, improving upon prior Shapley value methods.
We develop a new, principled algorithm for estimating the contribution of training data points to the behavior of a deep learning model, such as a specific prediction it makes. Our algorithm estimates the AME, a quantity that measures the expected (average) marginal effect of adding a data point to a subset of the training data, sampled from a given distribution. When subsets are sampled from the uniform distribution, the AME reduces to the well-known Shapley value. Our approach is inspired by causal inference and randomized experiments: we sample different subsets of the training data to train multiple submodels, and evaluate each submodel's behavior. We then use a LASSO regression to jointly estimate the AME of each data point, based on the subset compositions. Under sparsity assumptions ($k \ll N$ datapoints have large AME), our estimator requires only $O(k\log N)$ randomized submodel trainings, improving upon the best prior Shapley value estimators.