A Prior of a Googol Gaussians: a Tensor Ring Induced Prior for Generative Models
This work addresses the mode collapse and performance issues in generative models for domains such as text, image, video, and audio synthesis, offering a plug-and-play framework, though it is incremental as it builds on existing prior distribution enhancements.
The authors tackled the problem of limited prior distributions in generative models like GANs and VAEs by proposing a Tensor Ring Induced Prior (TRIP) that packs an exponential number of Gaussians with few parameters, resulting in improved Fréchet Inception Distance for GANs and Evidence Lower Bound for VAEs.
Generative models produce realistic objects in many domains, including text, image, video, and audio synthesis. Most popular models---Generative Adversarial Networks (GANs) and Variational Autoencoders (VAEs)---usually employ a standard Gaussian distribution as a prior. Previous works show that the richer family of prior distributions may help to avoid the mode collapse problem in GANs and to improve the evidence lower bound in VAEs. We propose a new family of prior distributions---Tensor Ring Induced Prior (TRIP)---that packs an exponential number of Gaussians into a high-dimensional lattice with a relatively small number of parameters. We show that these priors improve Fréchet Inception Distance for GANs and Evidence Lower Bound for VAEs. We also study generative models with TRIP in the conditional generation setup with missing conditions. Altogether, we propose a novel plug-and-play framework for generative models that can be utilized in any GAN and VAE-like architectures.