An Easy to Interpret Diagnostic for Approximate Inference: Symmetric Divergence Over Simulations
This addresses the need for reliable error estimation in approximate inference methods like variational inference, though it appears incremental as it builds on simulation-based diagnostics.
The paper tackles the problem of estimating errors for approximate probabilistic inference methods, which lack appropriate diagnostics, by introducing a diagnostic based on simulating datasets from the prior and estimating a symmetric KL-divergence.
It is important to estimate the errors of probabilistic inference algorithms. Existing diagnostics for Markov chain Monte Carlo methods assume inference is asymptotically exact, and are not appropriate for approximate methods like variational inference or Laplace's method. This paper introduces a diagnostic based on repeatedly simulating datasets from the prior and performing inference on each. The central observation is that it is possible to estimate a symmetric KL-divergence defined over these simulations.