Spectral Mixture Kernels for Bayesian Optimization
This work addresses a key bottleneck in Bayesian Optimization for expensive black-box optimization tasks, offering a method that balances computational speed with superior performance.
The paper tackles the challenge of selecting an appropriate probabilistic surrogate model in Bayesian Optimization by introducing a Gaussian Process-based method with spectral mixture kernels, achieving significant improvements in efficiency and optimization performance that outperform existing baselines across diverse synthetic and real-world problems.
Bayesian Optimization (BO) is a widely used approach for solving expensive black-box optimization tasks. However, selecting an appropriate probabilistic surrogate model remains an important yet challenging problem. In this work, we introduce a novel Gaussian Process (GP)-based BO method that incorporates spectral mixture kernels, derived from spectral densities formed by scale-location mixtures of Cauchy and Gaussian distributions. This method achieves a significant improvement in both efficiency and optimization performance, matching the computational speed of simpler kernels while delivering results that outperform more complex models and automatic BO methods. We provide bounds on the information gain and cumulative regret associated with obtaining the optimum. Extensive numerical experiments demonstrate that our method consistently outperforms existing baselines across a diverse range of synthetic and real-world problems, including both low- and high-dimensional settings.