Learning Entangled Single-Sample Gaussians in the Subset-of-Signals Model
This work addresses a statistical estimation problem in machine learning and data science, providing theoretical guarantees for a specific model, but it is incremental as it builds on existing bounds and extends lower bounds.
The paper tackles the problem of mean estimation for entangled single-sample Gaussians with common mean and different unknown variances, proposing a subset-of-signals model where some variances are bounded. It shows that an iterative truncated averaging method achieves error O(√(n ln n)/m) with high probability when m=Ω(√(n ln n)), matching existing bounds, and proves improved lower bounds for various ranges of m.
In the setting of entangled single-sample distributions, the goal is to estimate some common parameter shared by a family of $n$ distributions, given one single sample from each distribution. This paper studies mean estimation for entangled single-sample Gaussians that have a common mean but different unknown variances. We propose the subset-of-signals model where an unknown subset of $m$ variances are bounded by 1 while there are no assumptions on the other variances. In this model, we analyze a simple and natural method based on iteratively averaging the truncated samples, and show that the method achieves error $O \left(\frac{\sqrt{n\ln n}}{m}\right)$ with high probability when $m=Ω(\sqrt{n\ln n})$, matching existing bounds for this range of $m$. We further prove lower bounds, showing that the error is $Ω\left(\left(\frac{n}{m^4}\right)^{1/2}\right)$ when $m$ is between $Ω(\ln n)$ and $O(n^{1/4})$, and the error is $Ω\left(\left(\frac{n}{m^4}\right)^{1/6}\right)$ when $m$ is between $Ω(n^{1/4})$ and $O(n^{1 - ε})$ for an arbitrarily small $ε>0$, improving existing lower bounds and extending to a wider range of $m$.