MIRA: A Score for Conditional Distribution Accuracy and Model Comparison
For practitioners in Bayesian inference and model comparison, MIRA offers a practical tool to validate posterior distributions without computing marginal likelihood, though its effectiveness is shown only on toy problems.
MIRA is a sample-based score for evaluating conditional distribution accuracy using joint samples from the true data-generating process, enabling Bayesian model comparison without evidence computation. It provides theoretical reference values and uncertainty estimates, and is demonstrated on toy problems and Bayesian inference tasks.
We introduce Mira, a sample-based score for assessing the accuracy of a candidate conditional distribution using only joint samples from the true data-generating process. Relying on the principle that distributions coincide if they assign equal probability mass to all regions, we derive an analytic expression for the Mira statistic, whose average defines the Mira score. This formulation further allows us to compute theoretical reference values and uncertainty estimates when the candidate distribution matches the true one. This framework enables model comparison by quantifying the alignment between the conditional distribution of a candidate model and the true data generating process. Consequently, Mira enables Bayesian model comparison through direct posterior validation, bypassing the challenging evidence computation. We demonstrate its effectiveness across several toy problems and Bayesian inference tasks.