Yes, but Did It Work?: Evaluating Variational Inference
This addresses the challenge of detecting issues in variational approximations for practitioners in Bayesian statistics and machine learning, though it appears incremental as it builds on existing diagnostic methods.
The paper tackles the problem of evaluating variational inference approximations by proposing two diagnostic algorithms: Pareto-smoothed importance sampling (PSIS) for joint distribution goodness of fit and error improvement, and variational simulation-based calibration (VSBC) for assessing point estimate performance.
While it's always possible to compute a variational approximation to a posterior distribution, it can be difficult to discover problems with this approximation. We propose two diagnostic algorithms to alleviate this problem. The Pareto-smoothed importance sampling (PSIS) diagnostic gives a goodness of fit measurement for joint distributions, while simultaneously improving the error in the estimate. The variational simulation-based calibration (VSBC) assesses the average performance of point estimates.