Multi-Information Source Optimization
This addresses the problem of efficiently optimizing expensive functions using multiple biased approximations, which is incremental but important for applications like reinforcement learning and engineering.
The paper tackles Bayesian optimization of expensive black-box functions with cheaper, biased approximations by introducing a novel algorithm that rigorously handles model discrepancies and noisy observations. The method consistently outperforms state-of-the-art techniques, achieving higher objective values and lower exploration costs.
We consider Bayesian optimization of an expensive-to-evaluate black-box objective function, where we also have access to cheaper approximations of the objective. In general, such approximations arise in applications such as reinforcement learning, engineering, and the natural sciences, and are subject to an inherent, unknown bias. This model discrepancy is caused by an inadequate internal model that deviates from reality and can vary over the domain, making the utilization of these approximations a non-trivial task. We present a novel algorithm that provides a rigorous mathematical treatment of the uncertainties arising from model discrepancies and noisy observations. Its optimization decisions rely on a value of information analysis that extends the Knowledge Gradient factor to the setting of multiple information sources that vary in cost: each sampling decision maximizes the predicted benefit per unit cost. We conduct an experimental evaluation that demonstrates that the method consistently outperforms other state-of-the-art techniques: it finds designs of considerably higher objective value and additionally inflicts less cost in the exploration process.