Asymptotics of the Empirical Bootstrap Method Beyond Asymptotic Normality
This work provides a deeper theoretical understanding of the empirical bootstrap for statisticians and researchers working with non-asymptotically normal estimators, which is an incremental but important step.
This paper investigates the theoretical properties of the empirical bootstrap method for estimators that are not asymptotically normal. It establishes the limiting distribution, derives conditions for asymptotic consistency, and quantifies the convergence speed, while also proposing three alternative bootstrap methods for confidence intervals with coverage guarantees.
One of the most commonly used methods for forming confidence intervals for statistical inference is the empirical bootstrap, which is especially expedient when the limiting distribution of the estimator is unknown. However, despite its ubiquitous role, its theoretical properties are still not well understood for non-asymptotically normal estimators. In this paper, under stability conditions, we establish the limiting distribution of the empirical bootstrap estimator, derive tight conditions for it to be asymptotically consistent, and quantify the speed of convergence. Moreover, we propose three alternative ways to use the bootstrap method to build confidence intervals with coverage guarantees. Finally, we illustrate the generality and tightness of our results by a series of examples, including uniform confidence bands, two-sample kernel tests, minmax stochastic programs and the empirical risk of stacked estimators.