A Unified f-divergence Framework Generalizing VAE and GAN
This work addresses the problem of developing more flexible and unified deep generative models for researchers and practitioners, though it appears incremental as it builds upon existing encoder-decoder GAN architectures.
The paper introduces f-GM, a unified generative modeling framework that generalizes VAE and GAN by incorporating f-divergences, enabling flexible design of divergence functions without altering network structures or learning procedures. It allows for sampling, posterior inference, and likelihood evaluation, and experimentally explores effects like mode collapse and image sharpness with different f-divergence choices.
Developing deep generative models that flexibly incorporate diverse measures of probability distance is an important area of research. Here we develop an unified mathematical framework of f-divergence generative model, f-GM, that incorporates both VAE and f-GAN, and enables tractable learning with general f-divergences. f-GM allows the experimenter to flexibly design the f-divergence function without changing the structure of the networks or the learning procedure. f-GM jointly models three components: a generator, a inference network and a density estimator. Therefore it simultaneously enables sampling, posterior inference of the latent variable as well as evaluation of the likelihood of an arbitrary datum. f-GM belongs to the class of encoder-decoder GANs: our density estimator can be interpreted as playing the role of a discriminator between samples in the joint space of latent code and observed space. We prove that f-GM naturally simplifies to the standard VAE and to f-GAN as special cases, and illustrates the connections between different encoder-decoder GAN architectures. f-GM is compatible with general network architecture and optimizer. We leverage it to experimentally explore the effects -- e.g. mode collapse and image sharpness -- of different choices of f-divergence.