Batched Energy-Entropy acquisition for Bayesian Optimization
This work addresses the problem of efficient parallel point selection in Bayesian optimization for researchers and practitioners, presenting an incremental improvement over existing methods.
The authors tackled the challenge of high-dimensional and intractable acquisition functions in batched Bayesian optimization by proposing BEEBO, a statistical physics-inspired method that natively handles batches and generalizes to heteroskedastic problems, demonstrating competitive performance on various tasks.
Bayesian optimization (BO) is an attractive machine learning framework for performing sample-efficient global optimization of black-box functions. The optimization process is guided by an acquisition function that selects points to acquire in each round of BO. In batched BO, when multiple points are acquired in parallel, commonly used acquisition functions are often high-dimensional and intractable, leading to the use of sampling-based alternatives. We propose a statistical physics inspired acquisition function for BO with Gaussian processes that can natively handle batches. Batched Energy-Entropy acquisition for BO (BEEBO) enables tight control of the explore-exploit trade-off of the optimization process and generalizes to heteroskedastic black-box problems. We demonstrate the applicability of BEEBO on a range of problems, showing competitive performance to existing methods.