Efficient Estimation of Influence of a Training Instance
This work provides a more efficient way for researchers and practitioners to understand how individual training data points affect model predictions, which is an incremental improvement for model interpretability.
This paper addresses the challenge of efficiently estimating the influence of individual training instances on neural network predictions. The proposed method, inspired by dropout, allows for the estimation of influence by comparing sub-networks that have or have not learned specific training instances, demonstrating its ability to capture influences, improve error interpretability, and cleanse datasets for better generalization.
Understanding the influence of a training instance on a neural network model leads to improving interpretability. However, it is difficult and inefficient to evaluate the influence, which shows how a model's prediction would be changed if a training instance were not used. In this paper, we propose an efficient method for estimating the influence. Our method is inspired by dropout, which zero-masks a sub-network and prevents the sub-network from learning each training instance. By switching between dropout masks, we can use sub-networks that learned or did not learn each training instance and estimate its influence. Through experiments with BERT and VGGNet on classification datasets, we demonstrate that the proposed method can capture training influences, enhance the interpretability of error predictions, and cleanse the training dataset for improving generalization.