Thinking inside the box: A tutorial on grey-box Bayesian optimization
It provides a tutorial on grey-box BO methods for researchers and practitioners dealing with optimization problems where internal function details are available, representing an incremental advancement by systematizing existing approaches.
This tutorial addresses the problem of optimizing expensive-to-evaluate functions by leveraging internal information, such as partial observations or cheaper approximations, within Bayesian optimization (BO) to improve performance, though specific numerical results are not provided.
Bayesian optimization (BO) is a framework for global optimization of expensive-to-evaluate objective functions. Classical BO methods assume that the objective function is a black box. However, internal information about objective function computation is often available. For example, when optimizing a manufacturing line's throughput with simulation, we observe the number of parts waiting at each workstation, in addition to the overall throughput. Recent BO methods leverage such internal information to dramatically improve performance. We call these "grey-box" BO methods because they treat objective computation as partially observable and even modifiable, blending the black-box approach with so-called "white-box" first-principles knowledge of objective function computation. This tutorial describes these methods, focusing on BO of composite objective functions, where one can observe and selectively evaluate individual constituents that feed into the overall objective; and multi-fidelity BO, where one can evaluate cheaper approximations of the objective function by varying parameters of the evaluation oracle.