Meta-Thompson Sampling
This work addresses the challenge of exploration in bandits for online learning applications, presenting an incremental improvement over existing Thompson sampling methods.
The authors tackled the problem of efficient exploration in bandit settings by proposing Meta-Thompson Sampling (MetaTS), a variant that meta-learns an unknown prior to improve exploration, and demonstrated its effectiveness with theoretical analysis and empirical results showing quick adaptation.
Efficient exploration in bandits is a fundamental online learning problem. We propose a variant of Thompson sampling that learns to explore better as it interacts with bandit instances drawn from an unknown prior. The algorithm meta-learns the prior and thus we call it MetaTS. We propose several efficient implementations of MetaTS and analyze it in Gaussian bandits. Our analysis shows the benefit of meta-learning and is of a broader interest, because we derive a novel prior-dependent Bayes regret bound for Thompson sampling. Our theory is complemented by empirical evaluation, which shows that MetaTS quickly adapts to the unknown prior.