PMBO: Enhancing Black-Box Optimization through Multivariate Polynomial Surrogates
This is an incremental improvement for researchers and practitioners in optimization, offering a more robust and interpretable method for low-dimensional black-box problems.
The paper tackles black-box optimization by introducing PMBO, which combines polynomial approximation with Bayesian optimization, and shows it outperforms classic Bayesian optimization and performs comparably with state-of-the-art evolutionary algorithms like CMA-ES on low-dimensional problems.
We introduce a surrogate-based black-box optimization method, termed Polynomial-model-based optimization (PMBO). The algorithm alternates polynomial approximation with Bayesian optimization steps, using Gaussian processes to model the error between the objective and its polynomial fit. We describe the algorithmic design of PMBO and compare the results of the performance of PMBO with several optimization methods for a set of analytic test functions. The results show that PMBO outperforms the classic Bayesian optimization and is robust with respect to the choice of its correlation function family and its hyper-parameter setting, which, on the contrary, need to be carefully tuned in classic Bayesian optimization. Remarkably, PMBO performs comparably with state-of-the-art evolutionary algorithms such as the Covariance Matrix Adaptation -- Evolution Strategy (CMA-ES). This finding suggests that PMBO emerges as the pivotal choice among surrogate-based optimization methods when addressing low-dimensional optimization problems. Hereby, the simple nature of polynomials opens the opportunity for interpretation and analysis of the inferred surrogate model, providing a macroscopic perspective on the landscape of the objective function.