Information-Theoretic Bayesian Optimization for Bilevel Optimization Problems
This addresses a complex, understudied problem in optimization for researchers, but it is incremental as it builds on existing Bayesian optimization methods.
The paper tackles bilevel optimization with expensive black-box functions by proposing an information-theoretic Bayesian optimization approach that measures information gain for both upper- and lower-level problems, demonstrating effectiveness on benchmark datasets.
A bilevel optimization problem consists of two optimization problems nested as an upper- and a lower-level problem, in which the optimality of the lower-level problem defines a constraint for the upper-level problem. This paper considers Bayesian optimization (BO) for the case that both the upper- and lower-levels involve expensive black-box functions. Because of its nested structure, bilevel optimization has a complex problem definition and, compared with other standard extensions of BO such as multi-objective or constraint settings, it has not been widely studied. We propose an information-theoretic approach that considers the information gain of both the upper- and lower-optimal solutions and values. This enables us to define a unified criterion that measures the benefit for both level problems, simultaneously. Further, we also show a practical lower bound based approach to evaluating the information gain. We empirically demonstrate the effectiveness of our proposed method through several benchmark datasets.