Efficient Prior Selection in Gaussian Process Bandits with Thompson Sampling
This work addresses a practical bottleneck for practitioners in Bayesian optimization by providing theoretically-grounded methods for prior selection, though it is incremental as it builds on existing GP Thompson sampling frameworks.
The paper tackles the problem of selecting Gaussian process priors in bandit optimization when the prior is unknown, proposing two algorithms (PE-GP-TS and HP-GP-TS) that achieve theoretical regret bounds and demonstrate effectiveness in experiments with synthetic and real-world data.
Gaussian process (GP) bandits provide a powerful framework for performing blackbox optimization of unknown functions. The characteristics of the unknown function depends heavily on the assumed GP prior. Most work in the literature assume that this prior is known but in practice this seldom holds. Instead, practitioners often rely on maximum likelihood estimation to select the hyperparameters of the prior - which lacks theoretical guarantees. In this work, we propose two algorithms for joint prior selection and regret minimization in GP bandits based on GP Thompson sampling (GP-TS): Prior-Elimination GP-TS (PE-GP-TS) and HyperPrior GP-TS (HP-GP-TS). We theoretically analyze the algorithms and establish upper bounds for their respective regret. In addition, we demonstrate the effectiveness of our algorithms compared to the alternatives through experiments with synthetic and real-world data.