High-Order Error Bounds for Markovian LSA with Richardson-Romberg Extrapolation
This addresses bias reduction in stochastic approximation algorithms for machine learning and optimization, but it is incremental as it builds on existing methods like Richardson-Romberg extrapolation.
The paper tackled the bias in Linear Stochastic Approximation with Polyak-Ruppert averaging under Markovian noise by applying Richardson-Romberg extrapolation to cancel the leading linear-in-step-size bias term, showing that the leading error term aligns with the asymptotically optimal covariance matrix.
In this paper, we study the bias and high-order error bounds of the Linear Stochastic Approximation (LSA) algorithm with Polyak-Ruppert (PR) averaging under Markovian noise. We focus on the version of the algorithm with constant step size $α$ and propose a novel decomposition of the bias via a linearization technique. We analyze the structure of the bias and show that the leading-order term is linear in $α$ and cannot be eliminated by PR averaging. To address this, we apply the Richardson-Romberg (RR) extrapolation procedure, which effectively cancels the leading bias term. We derive high-order moment bounds for the RR iterates and show that the leading error term aligns with the asymptotically optimal covariance matrix of the vanilla averaged LSA iterates.