Advancing Bayesian Optimization: The Mixed-Global-Local (MGL) Kernel and Length-Scale Cool Down
This work addresses optimization challenges in fields like hyperparameter tuning and engineering design, but it is incremental as it builds on existing BO frameworks with specific enhancements.
The paper tackles the problem of improving Bayesian Optimization (BO) for expensive black-box optimization by focusing on online length-scale adaptation and kernel choice, proposing a method that uses the acquisition function for robust hyperparameter decisions and a Mixed-Global-Local kernel to integrate local convex optimization efficiency. The result shows improved efficiency compared to state-of-the-art BO methods on global optimization benchmarks.
Bayesian Optimization (BO) has become a core method for solving expensive black-box optimization problems. While much research focussed on the choice of the acquisition function, we focus on online length-scale adaption and the choice of kernel function. Instead of choosing hyperparameters in view of maximum likelihood on past data, we propose to use the acquisition function to decide on hyperparameter adaptation more robustly and in view of the future optimization progress. Further, we propose a particular kernel function that includes non-stationarity and local anisotropy and thereby implicitly integrates the efficiency of local convex optimization with global Bayesian optimization. Comparisons to state-of-the art BO methods underline the efficiency of these mechanisms on global optimization benchmarks.