MBORE: Multi-objective Bayesian Optimisation by Density-Ratio Estimation
This work addresses efficiency and scalability issues in optimization for researchers and practitioners dealing with expensive multi-objective functions, though it is incremental as it builds on prior density-ratio estimation methods.
The paper tackles the limitations of Gaussian processes in multi-objective Bayesian optimization, such as poor performance in high dimensions and high computational cost, by extending density-ratio estimation to this setting. The result is MBORE, which performs as well as or better than traditional methods on various benchmarks, with notable improvements in high-dimensional and real-world problems.
Optimisation problems often have multiple conflicting objectives that can be computationally and/or financially expensive. Mono-surrogate Bayesian optimisation (BO) is a popular model-based approach for optimising such black-box functions. It combines objective values via scalarisation and builds a Gaussian process (GP) surrogate of the scalarised values. The location which maximises a cheap-to-query acquisition function is chosen as the next location to expensively evaluate. While BO is an effective strategy, the use of GPs is limiting. Their performance decreases as the problem input dimensionality increases, and their computational complexity scales cubically with the amount of data. To address these limitations, we extend previous work on BO by density-ratio estimation (BORE) to the multi-objective setting. BORE links the computation of the probability of improvement acquisition function to that of probabilistic classification. This enables the use of state-of-the-art classifiers in a BO-like framework. In this work we present MBORE: multi-objective Bayesian optimisation by density-ratio estimation, and compare it to BO across a range of synthetic and real-world benchmarks. We find that MBORE performs as well as or better than BO on a wide variety of problems, and that it outperforms BO on high-dimensional and real-world problems.