Nonparametric Density Estimation under Adversarial Losses
This work addresses the theoretical understanding of convergence in density estimation for researchers in statistics and machine learning, particularly for GAN applications, but it is incremental as it extends existing minimax theory to adversarial losses.
The paper tackles the problem of determining minimax convergence rates for nonparametric density estimation under adversarial losses, including MMD, Wasserstein distance, and total variation, and finds that these rates depend on the loss function and density smoothness, with implications for GAN training and implicit generative models.
We study minimax convergence rates of nonparametric density estimation under a large class of loss functions called "adversarial losses", which, besides classical $\mathcal{L}^p$ losses, includes maximum mean discrepancy (MMD), Wasserstein distance, and total variation distance. These losses are closely related to the losses encoded by discriminator networks in generative adversarial networks (GANs). In a general framework, we study how the choice of loss and the assumed smoothness of the underlying density together determine the minimax rate. We also discuss implications for training GANs based on deep ReLU networks, and more general connections to learning implicit generative models in a minimax statistical sense.