Monotonic Alpha-divergence Minimisation for Variational Inference
This work addresses variational inference challenges for probabilistic modeling, offering incremental improvements over existing methods.
The paper tackles the problem of minimizing α-divergence in variational inference by introducing a family of iterative algorithms that ensure monotonic decrease in divergence, leading to improved results on multimodal target distributions and real data.
In this paper, we introduce a novel family of iterative algorithms which carry out $α$-divergence minimisation in a Variational Inference context. They do so by ensuring a systematic decrease at each step in the $α$-divergence between the variational and the posterior distributions. In its most general form, the variational distribution is a mixture model and our framework allows us to simultaneously optimise the weights and components parameters of this mixture model. Our approach permits us to build on various methods previously proposed for $α$-divergence minimisation such as Gradient or Power Descent schemes and we also shed a new light on an integrated Expectation Maximization algorithm. Lastly, we provide empirical evidence that our methodology yields improved results on several multimodal target distributions and on a real data example.