Warm Starting Bayesian Optimization
This work addresses a practical bottleneck for users of Bayesian optimization in business decision-making, but it is incremental as it adapts existing methods to a specific scenario.
The paper tackles the problem of reducing solution time for sequences of related optimization problems in Bayesian optimization, which lacks warm-starting capabilities, by developing a framework that builds a joint statistical model and uses value of information calculations, resulting in reduced computational effort.
We develop a framework for warm-starting Bayesian optimization, that reduces the solution time required to solve an optimization problem that is one in a sequence of related problems. This is useful when optimizing the output of a stochastic simulator that fails to provide derivative information, for which Bayesian optimization methods are well-suited. Solving sequences of related optimization problems arises when making several business decisions using one optimization model and input data collected over different time periods or markets. While many gradient-based methods can be warm started by initiating optimization at the solution to the previous problem, this warm start approach does not apply to Bayesian optimization methods, which carry a full metamodel of the objective function from iteration to iteration. Our approach builds a joint statistical model of the entire collection of related objective functions, and uses a value of information calculation to recommend points to evaluate.