Density Ratio Estimation-based Bayesian Optimization with Semi-Supervised Learning
This work addresses a specific challenge in Bayesian optimization for researchers and practitioners in science and engineering, but it is incremental as it builds on existing density ratio estimation methods.
The paper tackles the problem of overconfidence in supervised classifiers used for density ratio estimation-based Bayesian optimization by incorporating semi-supervised learning with unlabeled data, showing improved empirical results in scenarios with unlabeled point sampling and fixed-size pools.
Bayesian optimization has attracted huge attention from diverse research areas in science and engineering, since it is capable of efficiently finding a global optimum of an expensive-to-evaluate black-box function. In general, a probabilistic regression model is widely used as a surrogate function to model an explicit distribution over function evaluations given an input to estimate and a training dataset. Beyond the probabilistic regression-based methods, density ratio estimation-based Bayesian optimization has been suggested in order to estimate a density ratio of the groups relatively close and relatively far to a global optimum. Developing this line of research further, supervised classifiers are employed to estimate a class probability for the two groups instead of a density ratio. However, the supervised classifiers used in this strategy are prone to be overconfident for known knowledge on global solution candidates. Supposing that we have access to unlabeled points, e.g., predefined fixed-size pools, we propose density ratio estimation-based Bayesian optimization with semi-supervised learning to solve this challenge. Finally, we show the empirical results of our methods and several baseline methods in two distinct scenarios with unlabeled point sampling and a fixed-size pool, and analyze the validity of our methods in diverse experiments.