Maximin Robust Bayesian Experimental Design
This work addresses the problem of robust experimental design for researchers in Bayesian statistics and machine learning, offering a novel method to handle model misspecification, though it is incremental in building on existing information-theoretic concepts.
The paper tackles the brittleness of Bayesian experimental design under model misspecification by formulating it as a max-min game with adversarial nature, resulting in a robust objective based on Sibson's α-mutual information and Rényi divergence for belief updates. To address estimation challenges, it uses a PAC-Bayes framework to derive high-probability lower bounds on robust expected information gain, controlling finite-sample error.
We address the brittleness of Bayesian experimental design under model misspecification by formulating the problem as a max--min game between the experimenter and an adversarial nature subject to information-theoretic constraints. We demonstrate that this approach yields a robust objective governed by Sibson's $α$-mutual information~(MI), which identifies the $α$-tilted posterior as the robust belief update and establishes the Rényi divergence as the appropriate measure of conditional information gain. To mitigate the bias and variance of nested Monte Carlo estimators needed to estimate Sibson's $α$-MI, we adopt a PAC-Bayes framework to search over stochastic design policies, yielding rigorous high-probability lower bounds on the robust expected information gain that explicitly control finite-sample error.