Sequential Monte Carlo Bandits
This provides a flexible solution for various bandit problems, including restless and contextual bandits, with practical applications like advertising recommendation.
The authors tackled the problem of multi-armed bandits by proposing a framework combining sequential Monte Carlo with hierarchical Bayesian modeling, which outperformed other methods in simulations and showed increased efficacy in online video advertising recommendation.
In this paper we propose a flexible and efficient framework for handling multi-armed bandits, combining sequential Monte Carlo algorithms with hierarchical Bayesian modeling techniques. The framework naturally encompasses restless bandits, contextual bandits, and other bandit variants under a single inferential model. Despite the model's generality, we propose efficient Monte Carlo algorithms to make inference scalable, based on recent developments in sequential Monte Carlo methods. Through two simulation studies, the framework is shown to outperform other empirical methods, while also naturally scaling to more complex problems for which existing approaches can not cope. Additionally, we successfully apply our framework to online video-based advertising recommendation, and show its increased efficacy as compared to current state of the art bandit algorithms.