New Heuristics for Parallel and Scalable Bayesian Optimization
This work addresses the problem of making Bayesian optimization more accessible and effective for non-experts in parallel computing settings, though it appears incremental as it builds on existing methods.
The paper tackles the challenge of using Bayesian optimization in parallel computing environments by proposing practical heuristics to avoid pitfalls like oversampling edge parameters and over-exploitation of high-performance parameters, resulting in simple, open-source algorithms that are easily deployable.
Bayesian optimization has emerged as a strong candidate tool for global optimization of functions with expensive evaluation costs. However, due to the dynamic nature of research in Bayesian approaches, and the evolution of computing technology, using Bayesian optimization in a parallel computing environment remains a challenge for the non-expert. In this report, I review the state-of-the-art in parallel and scalable Bayesian optimization methods. In addition, I propose practical ways to avoid a few of the pitfalls of Bayesian optimization, such as oversampling of edge parameters and over-exploitation of high performance parameters. Finally, I provide relatively simple, heuristic algorithms, along with their open source software implementations, that can be immediately and easily deployed in any computing environment.