Importance Weighting and Variational Inference
This work addresses incremental improvements in variational inference methods for probabilistic modeling, relevant to researchers in machine learning and statistics.
The paper clarifies the applicability of importance sampling to variational inference, showing that Importance Weighted Variational Inference (IWVI) is an instance of augmented variational inference and identifies looseness in prior work, with experiments confirming its practicality. It also investigates inference with elliptical distributions, improving accuracy in low dimensions and convergence in high dimensions.
Recent work used importance sampling ideas for better variational bounds on likelihoods. We clarify the applicability of these ideas to pure probabilistic inference, by showing the resulting Importance Weighted Variational Inference (IWVI) technique is an instance of augmented variational inference, thus identifying the looseness in previous work. Experiments confirm IWVI's practicality for probabilistic inference. As a second contribution, we investigate inference with elliptical distributions, which improves accuracy in low dimensions, and convergence in high dimensions.