LABCAT: Locally adaptive Bayesian optimization using principal-component-aligned trust regions
This addresses the problem of optimizing expensive black-box functions for researchers and practitioners, but it is incremental as it builds on existing trust-region-based BO methods.
The paper tackles the limitations of Bayesian optimization (BO) for expensive black-box functions, such as computational slowdown and poor convergence, by proposing the LABCAT algorithm, which outperforms state-of-the-art BO and other black-box optimization methods in numerical experiments.
Bayesian optimization (BO) is a popular method for optimizing expensive black-box functions. BO has several well-documented shortcomings, including computational slowdown with longer optimization runs, poor suitability for non-stationary or ill-conditioned objective functions, and poor convergence characteristics. Several algorithms have been proposed that incorporate local strategies, such as trust regions, into BO to mitigate these limitations; however, none address all of them satisfactorily. To address these shortcomings, we propose the LABCAT algorithm, which extends trust-region-based BO by adding a rotation aligning the trust region with the weighted principal components and an adaptive rescaling strategy based on the length-scales of a local Gaussian process surrogate model with automatic relevance determination. Through extensive numerical experiments using a set of synthetic test functions and the well-known COCO benchmarking software, we show that the LABCAT algorithm outperforms several state-of-the-art BO and other black-box optimization algorithms.