A General Recipe for Likelihood-free Bayesian Optimization
This work addresses the problem of black-box optimization for researchers and practitioners by enabling more flexible and efficient optimization methods, though it is incremental as it builds on existing BO frameworks.
The paper tackles the limitation of Bayesian optimization (BO) by proposing likelihood-free BO (LFBO), which extends BO to a broader class of models and utilities without requiring tractable probabilistic surrogate models, and it outperforms state-of-the-art methods on real-world problems, with regret improvements of several orders of magnitude when leveraging composite structures.
The acquisition function, a critical component in Bayesian optimization (BO), can often be written as the expectation of a utility function under a surrogate model. However, to ensure that acquisition functions are tractable to optimize, restrictions must be placed on the surrogate model and utility function. To extend BO to a broader class of models and utilities, we propose likelihood-free BO (LFBO), an approach based on likelihood-free inference. LFBO directly models the acquisition function without having to separately perform inference with a probabilistic surrogate model. We show that computing the acquisition function in LFBO can be reduced to optimizing a weighted classification problem, where the weights correspond to the utility being chosen. By choosing the utility function for expected improvement (EI), LFBO outperforms various state-of-the-art black-box optimization methods on several real-world optimization problems. LFBO can also effectively leverage composite structures of the objective function, which further improves its regret by several orders of magnitude.