High-Dimensional Bayesian Optimization via Additive Models with Overlapping Groups
This work addresses a key open problem in Bayesian optimization for applications like parameter tuning and robotics, but it is incremental as it builds on prior additive models.
The paper tackles the challenge of scaling Bayesian optimization to high dimensions by generalizing additive models to allow overlapping subsets of variables, and demonstrates its effectiveness on synthetic and real-world data.
Bayesian optimization (BO) is a popular technique for sequential black-box function optimization, with applications including parameter tuning, robotics, environmental monitoring, and more. One of the most important challenges in BO is the development of algorithms that scale to high dimensions, which remains a key open problem despite recent progress. In this paper, we consider the approach of Kandasamy et al. (2015), in which the high-dimensional function decomposes as a sum of lower-dimensional functions on subsets of the underlying variables. In particular, we significantly generalize this approach by lifting the assumption that the subsets are disjoint, and consider additive models with arbitrary overlap among the subsets. By representing the dependencies via a graph, we deduce an efficient message passing algorithm for optimizing the acquisition function. In addition, we provide an algorithm for learning the graph from samples based on Gibbs sampling. We empirically demonstrate the effectiveness of our methods on both synthetic and real-world data.