Heavy-tailed Streaming Statistical Estimation
This addresses streaming statistical estimation under heavy-tailed distributions for applications requiring low space complexity, representing an incremental improvement with specific algorithmic refinements.
The paper tackles heavy-tailed statistical estimation in streaming settings by developing a clipped stochastic gradient descent algorithm with improved analysis under nuanced noise conditions, achieving exponential concentration guarantees with O(1) batch size and demonstrating applications in mean estimation and linear regression.
We consider the task of heavy-tailed statistical estimation given streaming $p$-dimensional samples. This could also be viewed as stochastic optimization under heavy-tailed distributions, with an additional $O(p)$ space complexity constraint. We design a clipped stochastic gradient descent algorithm and provide an improved analysis, under a more nuanced condition on the noise of the stochastic gradients, which we show is critical when analyzing stochastic optimization problems arising from general statistical estimation problems. Our results guarantee convergence not just in expectation but with exponential concentration, and moreover does so using $O(1)$ batch size. We provide consequences of our results for mean estimation and linear regression. Finally, we provide empirical corroboration of our results and algorithms via synthetic experiments for mean estimation and linear regression.