The Infinitesimal Jackknife and Combinations of Models
This provides a tool for statisticians and data scientists to assess uncertainty in ensemble methods, though it is an incremental extension of an existing technique.
The paper extended the Infinitesimal Jackknife method to estimate covariances between models, enabling uncertainty quantification for model combinations and comparisons, with validation through simulations and real data like Beijing Housing.
The Infinitesimal Jackknife is a general method for estimating variances of parametric models, and more recently also for some ensemble methods. In this paper we extend the Infinitesimal Jackknife to estimate the covariance between any two models. This can be used to quantify uncertainty for combinations of models, or to construct test statistics for comparing different models or ensembles of models fitted using the same training dataset. Specific examples in this paper use boosted combinations of models like random forests and M-estimators. We also investigate its application on neural networks and ensembles of XGBoost models. We illustrate the efficacy of variance estimates through extensive simulations and its application to the Beijing Housing data, and demonstrate the theoretical consistency of the Infinitesimal Jackknife covariance estimate.