Uniform Convergence Beyond Glivenko-Cantelli
This work addresses foundational theoretical challenges in statistical learning theory for researchers, providing incremental extensions to classical uniform convergence principles.
The paper tackles the problem of uniform mean estimation for collections of distributions on infinite binary sequences, extending beyond the empirical mean estimator to introduce Uniform Mean Estimability (UME-learnability). It shows that separability of mean vectors is sufficient but not necessary for UME-learnability, and proves that countable unions of UME-learnable collections are also UME-learnable, solving a conjecture.
We characterize conditions under which collections of distributions on $\{0,1\}^\mathbb{N}$ admit uniform estimation of their mean. Prior work from Vapnik and Chervonenkis (1971) has focused on uniform convergence using the empirical mean estimator, leading to the principle known as $P-$ Glivenko-Cantelli. We extend this framework by moving beyond the empirical mean estimator and introducing Uniform Mean Estimability, also called $UME-$ learnability, which captures when a collection permits uniform mean estimation by any arbitrary estimator. We work on the space created by the mean vectors of the collection of distributions. For each distribution, the mean vector records the expected value in each coordinate. We show that separability of the mean vectors is a sufficient condition for $UME-$ learnability. However, we show that separability of the mean vectors is not necessary for $UME-$ learnability by constructing a collection of distributions whose mean vectors are non-separable yet $UME-$ learnable using techniques fundamentally different from those used in our separability-based analysis. Finally, we establish that countable unions of $UME-$ learnable collections are also $UME-$ learnable, solving a conjecture posed in Cohen et al. (2025).