On the Robustness to Misspecification of $α$-Posteriors and Their Variational Approximations
This addresses robustness issues in Bayesian methods for practitioners dealing with model misspecification, though it is incremental as it builds on existing α-posterior frameworks.
The paper tackles the problem of parametric model misspecification in Bayesian inference by showing that tuning α-posteriors and their variational approximations reduces the Kullback-Leibler divergence from the true posterior, with the optimized divergence increasing logarithmically rather than linearly with misspecification severity.
$α$-posteriors and their variational approximations distort standard posterior inference by downweighting the likelihood and introducing variational approximation errors. We show that such distortions, if tuned appropriately, reduce the Kullback-Leibler (KL) divergence from the true, but perhaps infeasible, posterior distribution when there is potential parametric model misspecification. To make this point, we derive a Bernstein-von Mises theorem showing convergence in total variation distance of $α$-posteriors and their variational approximations to limiting Gaussian distributions. We use these distributions to evaluate the KL divergence between true and reported posteriors. We show this divergence is minimized by choosing $α$ strictly smaller than one, assuming there is a vanishingly small probability of model misspecification. The optimized value becomes smaller as the the misspecification becomes more severe. The optimized KL divergence increases logarithmically in the degree of misspecification and not linearly as with the usual posterior.