Optimal Mean Estimation without a Variance
This work provides a foundational understanding of optimal mean estimation for heavy-tailed distributions, which is crucial for robust statistical analysis in various scientific and engineering domains.
This paper addresses the problem of estimating the mean of heavy-tailed distributions where the variance does not exist, under a weak-moment assumption. The authors establish an information-theoretic lower bound for the optimal confidence interval and propose a computationally efficient estimator that achieves this bound.
We study the problem of heavy-tailed mean estimation in settings where the variance of the data-generating distribution does not exist. Concretely, given a sample $\mathbf{X} = \{X_i\}_{i = 1}^n$ from a distribution $\mathcal{D}$ over $\mathbb{R}^d$ with mean $μ$ which satisfies the following \emph{weak-moment} assumption for some ${α\in [0, 1]}$: \begin{equation*} \forall \|v\| = 1: \mathbb{E}_{X \thicksim \mathcal{D}}[\lvert \langle X - μ, v\rangle \rvert^{1 + α}] \leq 1, \end{equation*} and given a target failure probability, $δ$, our goal is to design an estimator which attains the smallest possible confidence interval as a function of $n,d,δ$. For the specific case of $α= 1$, foundational work of Lugosi and Mendelson exhibits an estimator achieving subgaussian confidence intervals, and subsequent work has led to computationally efficient versions of this estimator. Here, we study the case of general $α$, and establish the following information-theoretic lower bound on the optimal attainable confidence interval: \begin{equation*} Ω\left(\sqrt{\frac{d}{n}} + \left(\frac{d}{n}\right)^{\fracα{(1 + α)}} + \left(\frac{\log 1 / δ}{n}\right)^{\fracα{(1 + α)}}\right). \end{equation*} Moreover, we devise a computationally-efficient estimator which achieves this lower bound.