Agnostic Private Density Estimation for GMMs via List Global Stability
This work solves a privacy-preserving estimation problem for machine learning practitioners dealing with complex data distributions, representing an incremental advance by extending existing stability concepts to agnostic cases.
The paper tackles the problem of private density estimation for high-dimensional Gaussian mixture models in the agnostic setting, proving the first upper bound on sample complexity, which addresses a gap where only realizable settings were previously known.
We consider the problem of private density estimation for mixtures of unrestricted high dimensional Gaussians in the agnostic setting. We prove the first upper bound on the sample complexity of this problem. Previously, private learnability of high dimensional GMMs was only known in the realizable setting [Afzali et al., 2024]. To prove our result, we exploit the notion of $\textit{list global stability}$ [Ghazi et al., 2021b,a] that was originally introduced in the context of private supervised learning. We define an agnostic variant of this definition, showing that its existence is sufficient for agnostic private density estimation. We then construct an agnostic list globally stable learner for GMMs.