Regret-Based $(ε,δ)$-optimal Stopping Criteria for Bayesian Optimization
For practitioners using Bayesian optimization, this provides a principled way to stop optimization with formal guarantees, avoiding wasted computation.
This work develops theoretically sound stopping criteria for Bayesian optimization with GP-UCB, providing tighter regret bounds that guarantee an ε-optimal solution with high probability 1-δ upon termination, reducing unnecessary evaluations.
Bayesian optimization (BO) is a widely used iterative black-box optimization method that utilizes Gaussian process (GP) surrogate models. In practice, BO is typically terminated after a fixed evaluation budget is exhausted, which can incur unnecessary cost and provides no optimality guarantee on solution quality. Recent research in developing a practical stopping criterion has made empirical progress, yet a theoretically sound stopping criterion remains a work in progress. In this work, we present provably tighter instantaneous regret bounds for GP upper confidence bound (GP-UCB) at any given iteration. Then, we propose stopping criteria for GP-UCB based on this tighter bound that ensures an $ε$-optimal solution with high probability $1-δ$ upon termination. Numerical experiments are performed to validate and demonstrate the effectiveness and efficiency of our stopping criteria.