LED: Latent Variable-based Estimation of Density
This addresses the trade-off in generative models for machine learning practitioners, offering a hybrid solution that is incremental in combining existing approaches.
The paper tackles the problem of generative models that either produce high-quality samples without exact density estimation or provide exact density estimation at the cost of sample quality, by proposing LED, a model that enables both efficient sampling and accurate density estimation, with experimental results showing accurate density estimates and good sample quality across various datasets.
Modern generative models are roughly divided into two main categories: (1) models that can produce high-quality random samples, but cannot estimate the exact density of new data points and (2) those that provide exact density estimation, at the expense of sample quality and compactness of the latent space. In this work we propose LED, a new generative model closely related to GANs, that allows not only efficient sampling but also efficient density estimation. By maximizing log-likelihood on the output of the discriminator, we arrive at an alternative adversarial optimization objective that encourages generated data diversity. This formulation provides insights into the relationships between several popular generative models. Additionally, we construct a flow-based generator that can compute exact probabilities for generated samples, while allowing low-dimensional latent variables as input. Our experimental results, on various datasets, show that our density estimator produces accurate estimates, while retaining good quality in the generated samples.