Robust Estimation for Nonparametric Families via Generative Adversarial Networks
This work addresses robust statistics problems for high-dimensional data in machine learning, offering a more general framework than prior Gaussian-focused methods, though it builds incrementally on existing GAN approaches.
The paper tackles robust estimation for nonparametric families with adversarially corrupted samples by extending GAN-based methods to handle distributions with bounded Orlicz norms, achieving results for mean, second moment, and linear regression without requiring specific distributional assumptions.
We provide a general framework for designing Generative Adversarial Networks (GANs) to solve high dimensional robust statistics problems, which aim at estimating unknown parameter of the true distribution given adversarially corrupted samples. Prior work focus on the problem of robust mean and covariance estimation when the true distribution lies in the family of Gaussian distributions or elliptical distributions, and analyze depth or scoring rule based GAN losses for the problem. Our work extend these to robust mean estimation, second moment estimation, and robust linear regression when the true distribution only has bounded Orlicz norms, which includes the broad family of sub-Gaussian, sub-Exponential and bounded moment distributions. We also provide a different set of sufficient conditions for the GAN loss to work: we only require its induced distance function to be a cumulative density function of some light-tailed distribution, which is easily satisfied by neural networks with sigmoid activation. In terms of techniques, our proposed GAN losses can be viewed as a smoothed and generalized Kolmogorov-Smirnov distance, which overcomes the computational intractability of the original Kolmogorov-Smirnov distance used in the prior work.