Estimation of multiple mean vectors in high dimension
This work addresses the challenge of improving estimation accuracy for high-dimensional data in statistical learning, though it appears incremental as it builds on existing convex combination techniques.
The paper tackles the problem of estimating multiple high-dimensional mean vectors from independent samples by proposing estimators based on convex combinations of empirical means with data-dependent weights, and shows that these methods asymptotically approach an oracle improvement in quadratic risk as the effective dimension increases.
We endeavour to estimate numerous multi-dimensional means of various probability distributions on a common space based on independent samples. Our approach involves forming estimators through convex combinations of empirical means derived from these samples. We introduce two strategies to find appropriate data-dependent convex combination weights: a first one employing a testing procedure to identify neighbouring means with low variance, which results in a closed-form plug-in formula for the weights, and a second one determining weights via minimization of an upper confidence bound on the quadratic risk. Through theoretical analysis, we evaluate the improvement in quadratic risk offered by our methods compared to the empirical means. Our analysis focuses on a dimensional asymptotics perspective, showing that our methods asymptotically approach an oracle (minimax) improvement as the effective dimension of the data increases. We demonstrate the efficacy of our methods in estimating multiple kernel mean embeddings through experiments on both simulated and real-world datasets.