Active Bayesian Optimization: Minimizing Minimizer Entropy
This work addresses the challenge of active optimization for researchers and practitioners in machine learning, offering a novel approach that is incremental in improving upon existing Bayesian optimization methods.
The paper tackles the problem of active optimization by proposing a new criterion that reduces uncertainty in the function minimizer's posterior distribution, demonstrating that it accurately obtains the global minimizer compared to conventional Bayesian optimization criteria.
The ultimate goal of optimization is to find the minimizer of a target function.However, typical criteria for active optimization often ignore the uncertainty about the minimizer. We propose a novel criterion for global optimization and an associated sequential active learning strategy using Gaussian processes.Our criterion is the reduction of uncertainty in the posterior distribution of the function minimizer. It can also flexibly incorporate multiple global minimizers. We implement a tractable approximation of the criterion and demonstrate that it obtains the global minimizer accurately compared to conventional Bayesian optimization criteria.