High-accuracy log-concave sampling with stochastic queries
This work addresses the efficiency of sampling algorithms for log-concave distributions, which is a fundamental problem in machine learning and statistics, by providing theoretical insights into the impact of stochasticity on query complexity.
The paper tackles the problem of log-concave sampling by showing that high-accuracy guarantees with poly-logarithmic scaling in target accuracy are achievable using stochastic gradients with subexponential tails, and it demonstrates a separation from convex optimization where such scaling is not possible. It also proves that light-tailed stochastic gradients are necessary for high accuracy, with minimax-optimal query complexity scaling as Θ(1/δ) in the bounded variance case.
We show that high-accuracy guarantees for log-concave sampling -- that is, iteration and query complexities which scale as $\mathrm{poly}\log(1/δ)$, where $δ$ is the desired target accuracy -- are achievable using stochastic gradients with subexponential tails. Notably, this exhibits a separation with the problem of convex optimization, where stochasticity (even additive Gaussian noise) in the gradient oracle incurs $\mathrm{poly}(1/δ)$ queries. We also give an information-theoretic argument that light-tailed stochastic gradients are necessary for high accuracy: for example, in the bounded variance case, we show that the minimax-optimal query complexity scales as $Θ(1/δ)$. Our framework also provides similar high accuracy guarantees under stochastic zeroth order (value) queries.