Approximation-Aware Bayesian Optimization
This addresses computational bottlenecks in high-dimensional Bayesian optimization for applications like molecular design, though it appears incremental as it modifies existing SVGPs rather than introducing a fundamentally new approach.
The paper tackles the problem of suboptimal data acquisition in high-dimensional Bayesian optimization caused by sparse variational Gaussian process approximations, and presents a method that unifies GP approximation and data acquisition into joint optimization to outperform standard SVGPs on benchmark tasks.
High-dimensional Bayesian optimization (BO) tasks such as molecular design often require 10,000 function evaluations before obtaining meaningful results. While methods like sparse variational Gaussian processes (SVGPs) reduce computational requirements in these settings, the underlying approximations result in suboptimal data acquisitions that slow the progress of optimization. In this paper we modify SVGPs to better align with the goals of BO: targeting informed data acquisition rather than global posterior fidelity. Using the framework of utility-calibrated variational inference, we unify GP approximation and data acquisition into a joint optimization problem, thereby ensuring optimal decisions under a limited computational budget. Our approach can be used with any decision-theoretic acquisition function and is compatible with trust region methods like TuRBO. We derive efficient joint objectives for the expected improvement and knowledge gradient acquisition functions in both the standard and batch BO settings. Our approach outperforms standard SVGPs on high-dimensional benchmark tasks in control and molecular design.