Revisiting Methods for Finding Influential Examples
This work addresses the reliability of explainability methods for machine learning practitioners, highlighting a critical flaw in current approaches and offering a practical improvement, though it is incremental in nature.
The paper demonstrates that existing instance-based explainability methods for finding influential training examples are unstable due to sensitivity to initialization and data ordering, and shows that standard evaluation metrics like LOO influence are poor, proposing poisoning attack detection as a better alternative. It introduces a simple baseline that significantly improves these methods on downstream tasks, with unspecified but 'very significant' gains.
Several instance-based explainability methods for finding influential training examples for test-time decisions have been proposed recently, including Influence Functions, TraceIn, Representer Point Selection, Grad-Dot, and Grad-Cos. Typically these methods are evaluated using LOO influence (Cook's distance) as a gold standard, or using various heuristics. In this paper, we show that all of the above methods are unstable, i.e., extremely sensitive to initialization, ordering of the training data, and batch size. We suggest that this is a natural consequence of how in the literature, the influence of examples is assumed to be independent of model state and other examples -- and argue it is not. We show that LOO influence and heuristics are, as a result, poor metrics to measure the quality of instance-based explanations, and instead propose to evaluate such explanations by their ability to detect poisoning attacks. Further, we provide a simple, yet effective baseline to improve all of the above methods and show how it leads to very significant improvements on downstream tasks.