Advances in Variational Inference
It addresses the problem of efficient and flexible approximate inference for researchers and practitioners in machine learning, but it is incremental as it reviews existing advances rather than introducing novel contributions.
This review paper tackles the challenge of approximating intractable Bayesian posteriors in machine learning by providing an overview of recent trends in variational inference, including scalable, generic, accurate, and amortized methods, without presenting new experimental results or concrete numbers.
Many modern unsupervised or semi-supervised machine learning algorithms rely on Bayesian probabilistic models. These models are usually intractable and thus require approximate inference. Variational inference (VI) lets us approximate a high-dimensional Bayesian posterior with a simpler variational distribution by solving an optimization problem. This approach has been successfully used in various models and large-scale applications. In this review, we give an overview of recent trends in variational inference. We first introduce standard mean field variational inference, then review recent advances focusing on the following aspects: (a) scalable VI, which includes stochastic approximations, (b) generic VI, which extends the applicability of VI to a large class of otherwise intractable models, such as non-conjugate models, (c) accurate VI, which includes variational models beyond the mean field approximation or with atypical divergences, and (d) amortized VI, which implements the inference over local latent variables with inference networks. Finally, we provide a summary of promising future research directions.