Variational f-divergence Minimization
This addresses a gap in machine learning for researchers and practitioners who need flexible training criteria, such as for sharp image generation, though it appears incremental as it extends existing variational methods.
The paper tackles the problem of training probabilistic latent variable models using f-divergences beyond maximum likelihood, and presents a variational method combined with Spread Divergence to enable training with any f-divergence.
Probabilistic models are often trained by maximum likelihood, which corresponds to minimizing a specific f-divergence between the model and data distribution. In light of recent successes in training Generative Adversarial Networks, alternative non-likelihood training criteria have been proposed. Whilst not necessarily statistically efficient, these alternatives may better match user requirements such as sharp image generation. A general variational method for training probabilistic latent variable models using maximum likelihood is well established; however, how to train latent variable models using other f-divergences is comparatively unknown. We discuss a variational approach that, when combined with the recently introduced Spread Divergence, can be applied to train a large class of latent variable models using any f-divergence.