Bayesian Experimental Design via Contrastive Diffusions
This work addresses the scalability issue in BOED for researchers and practitioners in fields like experimental design and machine learning, offering a novel method to handle complex scenarios previously impractical, though it is incremental in improving computational efficiency.
The authors tackled the computational complexity of Bayesian Optimal Experimental Design (BOED) in high-dimensional settings by introducing a pooled posterior distribution and a new gradient expression for Expected Information Gain, enabling efficient optimization via diffusion-based samplers and bi-level optimization, resulting in significant efficiency gains that extend BOED to diffusion models.
Bayesian Optimal Experimental Design (BOED) is a powerful tool to reduce the cost of running a sequence of experiments. When based on the Expected Information Gain (EIG), design optimization corresponds to the maximization of some intractable expected contrast between prior and posterior distributions. Scaling this maximization to high dimensional and complex settings has been an issue due to BOED inherent computational complexity. In this work, we introduce a pooled posterior distribution with cost-effective sampling properties and provide a tractable access to the EIG contrast maximization via a new EIG gradient expression. Diffusion-based samplers are used to compute the dynamics of the pooled posterior and ideas from bi-level optimization are leveraged to derive an efficient joint sampling-optimization loop. The resulting efficiency gain allows to extend BOED to the well-tested generative capabilities of diffusion models. By incorporating generative models into the BOED framework, we expand its scope and its use in scenarios that were previously impractical. Numerical experiments and comparison with state-of-the-art methods show the potential of the approach.