The Approximate Fisher Influence Function: Faster Estimation of Data Influence in Statistical Models
This work addresses the need for faster and more robust influence estimation in machine learning models, which is crucial for tasks like debugging and improving models, though it appears incremental as it builds on existing influence function-based methods.
The paper tackles the problem of quantifying how small changes in training data affect model performance by reformulating it as a weighted empirical risk minimization and using information geometry to derive a new algorithm. The result is a method that is versatile across applications, remains informative in non-convex cases, and offers significant computational advantages over existing Newton step-based methods.
Quantifying the influence of infinitesimal changes in training data on model performance is crucial for understanding and improving machine learning models. In this work, we reformulate this problem as a weighted empirical risk minimization and enhance existing influence function-based methods by using information geometry to derive a new algorithm to estimate influence. Our formulation proves versatile across various applications, and we further demonstrate in simulations how it remains informative even in non-convex cases. Furthermore, we show that our method offers significant computational advantages over current Newton step-based methods.