TS-RSR: A provably efficient approach for batch Bayesian Optimization
This work addresses the challenge of efficient batch sampling in Bayesian Optimization for applications like hyperparameter tuning, though it appears incremental as it builds on existing Thompson Sampling and regret-based approaches.
The paper tackles the problem of batch Bayesian Optimization by introducing TS-RSR, a method that coordinates actions within each batch to minimize redundancy while focusing on high predictive means or uncertainty, and demonstrates state-of-the-art performance on synthetic and realistic test functions.
This paper presents a new approach for batch Bayesian Optimization (BO) called Thompson Sampling-Regret to Sigma Ratio directed sampling (TS-RSR), where we sample a new batch of actions by minimizing a Thompson Sampling approximation of a regret to uncertainty ratio. Our sampling objective is able to coordinate the actions chosen in each batch in a way that minimizes redundancy between points whilst focusing on points with high predictive means or high uncertainty. Theoretically, we provide rigorous convergence guarantees on our algorithm's regret, and numerically, we demonstrate that our method attains state-of-the-art performance on a range of challenging synthetic and realistic test functions, where it outperforms several competitive benchmark batch BO algorithms.