Improved Central Limit Theorem and Bootstrap Approximations for Linear Stochastic Approximation
This work provides incremental improvements in theoretical guarantees for statistical inference in stochastic approximation algorithms, relevant for researchers in optimization and statistics.
The paper refines Berry-Esseen bounds for normal approximation of Polyak-Ruppert averaged iterates in linear stochastic approximation, achieving rates up to order n^{-1/3} in convex distance, and proves non-asymptotic validity of a multiplier bootstrap with rates up to 1/√n, improving upon prior results.
In this paper, we refine the Berry-Esseen bounds for the multivariate normal approximation of Polyak-Ruppert averaged iterates arising from the linear stochastic approximation (LSA) algorithm with decreasing step size. We consider the normal approximation by the Gaussian distribution with covariance matrix predicted by the Polyak-Juditsky central limit theorem and establish the rate up to order $n^{-1/3}$ in convex distance, where $n$ is the number of samples used in the algorithm. We also prove a non-asymptotic validity of the multiplier bootstrap procedure for approximating the distribution of the rescaled error of the averaged LSA estimator. We establish approximation rates of order up to $1/\sqrt{n}$ for the latter distribution, which significantly improves upon the previous results obtained by Samsonov et al. (2024).