Sampling-Based Accuracy Testing of Posterior Estimators for General Inference
This addresses the challenge of validating posterior estimators for parameter inference in scientific disciplines, offering a novel testing approach that is necessary and sufficient for accuracy.
The paper tackles the problem of assessing the accuracy of posterior distributions estimated by generative models, introducing TARP coverage testing as a method that does not require posterior evaluations. It demonstrates the method on synthetic examples, showing it can detect inaccuracies in high-dimensional spaces where existing methods fail.
Parameter inference, i.e. inferring the posterior distribution of the parameters of a statistical model given some data, is a central problem to many scientific disciplines. Generative models can be used as an alternative to Markov Chain Monte Carlo methods for conducting posterior inference, both in likelihood-based and simulation-based problems. However, assessing the accuracy of posteriors encoded in generative models is not straightforward. In this paper, we introduce `Tests of Accuracy with Random Points' (TARP) coverage testing as a method to estimate coverage probabilities of generative posterior estimators. Our method differs from previously-existing coverage-based methods, which require posterior evaluations. We prove that our approach is necessary and sufficient to show that a posterior estimator is accurate. We demonstrate the method on a variety of synthetic examples, and show that TARP can be used to test the results of posterior inference analyses in high-dimensional spaces. We also show that our method can detect inaccurate inferences in cases where existing methods fail.