MALIBO: Meta-learning for Likelihood-free Bayesian Optimization
This work improves Bayesian optimization for researchers and practitioners by enabling faster and more reliable adaptation to new tasks, though it is incremental as it builds on existing meta-learning BO methods.
The paper tackled the problem of meta-learning for Bayesian optimization by addressing scalability, sensitivity to observation differences, and task uncertainty, resulting in a method that outperforms state-of-the-art approaches in benchmarks with strong anytime performance.
Bayesian optimization (BO) is a popular method to optimize costly black-box functions. While traditional BO optimizes each new target task from scratch, meta-learning has emerged as a way to leverage knowledge from related tasks to optimize new tasks faster. However, existing meta-learning BO methods rely on surrogate models that suffer from scalability issues and are sensitive to observations with different scales and noise types across tasks. Moreover, they often overlook the uncertainty associated with task similarity. This leads to unreliable task adaptation when only limited observations are obtained or when the new tasks differ significantly from the related tasks. To address these limitations, we propose a novel meta-learning BO approach that bypasses the surrogate model and directly learns the utility of queries across tasks. Our method explicitly models task uncertainty and includes an auxiliary model to enable robust adaptation to new tasks. Extensive experiments show that our method demonstrates strong anytime performance and outperforms state-of-the-art meta-learning BO methods in various benchmarks.