Taming Non-stationary Bandits: A Bayesian Approach
This addresses the challenge of adapting bandit algorithms to changing environments, which is incremental as it builds on existing Bayesian and Thompson Sampling methods.
The paper tackled the multi-armed bandit problem in non-stationary environments by proposing a Bayesian variant of Thompson Sampling that reduces past observation effects and includes an optimistic version, achieving validated utility through extensive empirical analysis and comparison with state-of-the-art algorithms.
We consider the multi armed bandit problem in non-stationary environments. Based on the Bayesian method, we propose a variant of Thompson Sampling which can be used in both rested and restless bandit scenarios. Applying discounting to the parameters of prior distribution, we describe a way to systematically reduce the effect of past observations. Further, we derive the exact expression for the probability of picking sub-optimal arms. By increasing the exploitative value of Bayes' samples, we also provide an optimistic version of the algorithm. Extensive empirical analysis is conducted under various scenarios to validate the utility of proposed algorithms. A comparison study with various state-of-the-arm algorithms is also included.