Learning the Base Distribution in Implicit Generative Models
This addresses the problem of modeling complex data distributions more accurately for generative modeling tasks, though it appears incremental as it builds on existing auto-encoder frameworks.
The paper tackles the limitation of simple latent distributions in generative models by proposing a two-stage optimization method that learns complex latent distributions, resulting in improved performance over GANs and VAEs on MNIST and CELEB-A image datasets, with extensions to speech and music generation.
Popular generative model learning methods such as Generative Adversarial Networks (GANs), and Variational Autoencoders (VAE) enforce the latent representation to follow simple distributions such as isotropic Gaussian. In this paper, we argue that learning a complicated distribution over the latent space of an auto-encoder enables more accurate modeling of complicated data distributions. Based on this observation, we propose a two stage optimization procedure which maximizes an approximate implicit density model. We experimentally verify that our method outperforms GANs and VAEs on two image datasets (MNIST, CELEB-A). We also show that our approach is amenable to learning generative model for sequential data, by learning to generate speech and music.