Practical Bayesian Optimization of Objectives with Conditioning Variables
This work addresses the need for efficient optimization in scenarios with multiple conditional objectives, such as in healthcare or machine learning, though it is incremental as it builds on existing Bayesian optimization methods.
The authors tackled the problem of optimizing multiple related objectives conditioned on state variables, such as tuning CNN hyperparameters for different data partitions, by proposing the ConBO framework with a Hybrid Knowledge Gradient acquisition function. The method performed similarly to or significantly better than recent works across various problems and is easily parallelized.
Bayesian optimization is a class of data efficient model based algorithms typically focused on global optimization. We consider the more general case where a user is faced with multiple problems that each need to be optimized conditional on a state variable, for example given a range of cities with different patient distributions, we optimize the ambulance locations conditioned on patient distribution. Given partitions of CIFAR-10, we optimize CNN hyperparameters for each partition. Similarity across objectives boosts optimization of each objective in two ways: in modelling by data sharing across objectives, and also in acquisition by quantifying how a single point on one objective can provide benefit to all objectives. For this we propose a framework for conditional optimization: ConBO. This can be built on top of a range of acquisition functions and we propose a new Hybrid Knowledge Gradient acquisition function. The resulting method is intuitive and theoretically grounded, performs either similar to or significantly better than recently published works on a range of problems, and is easily parallelized to collect a batch of points.