Bayesian Optimization of Bilevel Problems
This addresses complex decision-making problems in fields like economics and engineering, but it appears incremental as it builds on existing Bayesian Optimization approaches for bilevel problems.
The paper tackles bilevel optimization with black-box, expensive-to-evaluate functions by proposing a Bayesian Optimization framework that models functions as Gaussian processes and introduces a novel acquisition function, resulting in high sample efficiency and outperforming existing methods.
Bilevel optimization, a hierarchical mathematical framework where one optimization problem is nested within another, has emerged as a powerful tool for modeling complex decision-making processes in various fields such as economics, engineering, and machine learning. This paper focuses on bilevel optimization where both upper-level and lower-level functions are black boxes and expensive to evaluate. We propose a Bayesian Optimization framework that models the upper and lower-level functions as Gaussian processes over the combined space of upper and lower-level decisions, allowing us to exploit knowledge transfer between different sub-problems. Additionally, we propose a novel acquisition function for this model. Our experimental results demonstrate that the proposed algorithm is highly sample-efficient and outperforms existing methods in finding high-quality solutions.