HiBBO: HiPPO-based Space Consistency for High-dimensional Bayesian Optimisation
This work addresses the challenge of scaling Bayesian Optimization to high-dimensional problems for applications like neural architecture search and materials science, though it is incremental as it builds on existing VAE-based approaches.
The paper tackled the problem of Bayesian Optimization (BO) becoming less effective in high-dimensional spaces due to functional distribution mismatch in Variational Autoencoder (VAE)-based methods, and proposed HiBBO, a framework using HiPPO to improve space consistency, which outperformed existing VAE-BO methods in convergence speed and solution quality on benchmark tasks.
Bayesian Optimisation (BO) is a powerful tool for optimising expensive blackbox functions but its effectiveness diminishes in highdimensional spaces due to sparse data and poor surrogate model scalability While Variational Autoencoder (VAE) based approaches address this by learning low-dimensional latent representations the reconstructionbased objective function often brings the functional distribution mismatch between the latent space and original space leading to suboptimal optimisation performance In this paper we first analyse the reason why reconstructiononly loss may lead to distribution mismatch and then propose HiBBO a novel BO framework that introduces the space consistency into the latent space construction in VAE using HiPPO - a method for longterm sequence modelling - to reduce the functional distribution mismatch between the latent space and original space Experiments on highdimensional benchmark tasks demonstrate that HiBBO outperforms existing VAEBO methods in convergence speed and solution quality Our work bridges the gap between high-dimensional sequence representation learning and efficient Bayesian Optimisation enabling broader applications in neural architecture search materials science and beyond.