Exponential Tilting of Generative Models: Improving Sample Quality by Training and Sampling from Latent Energy
This method addresses the issue of poor sample quality in generative models like normalizing flows and VAEs, offering a general and efficient solution for practitioners, though it appears incremental as it builds on existing models.
The paper tackles the problem of improving sample quality in pre-trained likelihood-based generative models by introducing an energy function on the latent space, which is trained to maximize data likelihood and used to generate new samples, resulting in significant quality enhancements with minimal computational cost.
In this paper, we present a general method that can improve the sample quality of pre-trained likelihood based generative models. Our method constructs an energy function on the latent variable space that yields an energy function on samples produced by the pre-trained generative model. The energy based model is efficiently trained by maximizing the data likelihood, and after training, new samples in the latent space are generated from the energy based model and passed through the generator to producing samples in observation space. We show that using our proposed method, we can greatly improve the sample quality of popular likelihood based generative models, such as normalizing flows and VAEs, with very little computational overhead.