FigBO: A Generalized Acquisition Function Framework with Look-Ahead Capability for Bayesian Optimization
This work addresses a bottleneck in Bayesian optimization for researchers and practitioners by improving optimization efficiency, though it is incremental as it builds on existing myopic methods.
The paper tackles the limitation of myopic acquisition functions in Bayesian optimization by proposing FigBO, a generalized acquisition function with look-ahead capability, which achieves state-of-the-art performance and significantly faster convergence across diverse tasks.
Bayesian optimization is a powerful technique for optimizing expensive-to-evaluate black-box functions, consisting of two main components: a surrogate model and an acquisition function. In recent years, myopic acquisition functions have been widely adopted for their simplicity and effectiveness. However, their lack of look-ahead capability limits their performance. To address this limitation, we propose FigBO, a generalized acquisition function that incorporates the future impact of candidate points on global information gain. FigBO is a plug-and-play method that can integrate seamlessly with most existing myopic acquisition functions. Theoretically, we analyze the regret bound and convergence rate of FigBO when combined with the myopic base acquisition function expected improvement (EI), comparing them to those of standard EI. Empirically, extensive experimental results across diverse tasks demonstrate that FigBO achieves state-of-the-art performance and significantly faster convergence compared to existing methods.