Optimizing Bayesian acquisition functions in Gaussian Processes
This work provides incremental improvements in Bayesian Optimization methods for researchers and practitioners in machine learning and optimization.
The paper analyzed various Bayesian acquisition functions and optimizers for Gaussian Processes to improve global optimization of unknown objective functions, finding that the choice of initial sample positions significantly impacts performance.
Bayesian Optimization is an effective method for searching the global maxima of an objective function especially if the function is unknown. The process comprises of using a surrogate function and choosing an acquisition function followed by optimizing the acquisition function to find the next sampling point. This paper analyzes different acquistion functions like Maximum Probability of Improvement and Expected Improvement and various optimizers like L-BFGS and TNC to optimize the acquisitions functions for finding the next sampling point. Along with the analysis of time taken, the paper also shows the importance of position of initial samples chosen.