Optimization, fast and slow: optimally switching between local and Bayesian optimization
This addresses optimization efficiency for researchers and practitioners in fields like machine learning and engineering, though it appears incremental as it builds on existing Bayesian and local optimization methods.
The paper tackles the problem of efficiently finding global minima in optimization by developing BLOSSOM, the first Bayesian Optimization algorithm that dynamically switches between multiple acquisition functions and local optimization at each step. The result is superior convergence to the global minimum on tested problems while efficiently using function evaluations, with a novel stopping condition based on expected regret.
We develop the first Bayesian Optimization algorithm, BLOSSOM, which selects between multiple alternative acquisition functions and traditional local optimization at each step. This is combined with a novel stopping condition based on expected regret. This pairing allows us to obtain the best characteristics of both local and Bayesian optimization, making efficient use of function evaluations while yielding superior convergence to the global minimum on a selection of optimization problems, and also halting optimization once a principled and intuitive stopping condition has been fulfilled.