Exploiting Adjacent Similarity in Multi-Armed Bandit Tasks via Transfer of Reward Samples
This work addresses the challenge of improving regret in multi-task bandit problems for scenarios where tasks are similar, offering incremental algorithmic enhancements.
The paper tackles the problem of sequential multi-task learning in stochastic multi-armed bandits by exploiting adjacent similarity between tasks to transfer reward samples, resulting in reduced regret compared to no transfer and naive transfer methods, with empirical results showing performance improvements.
We consider a sequential multi-task problem, where each task is modeled as the stochastic multi-armed bandit with K arms. We assume the bandit tasks are adjacently similar in the sense that the difference between the mean rewards of the arms for any two consecutive tasks is bounded by a parameter. We propose two algorithms (one assumes the parameter is known while the other does not) based on UCB to transfer reward samples from preceding tasks to improve the overall regret across all tasks. Our analysis shows that transferring samples reduces the regret as compared to the case of no transfer. We provide empirical results for our algorithms, which show performance improvement over the standard UCB algorithm without transfer and a naive transfer algorithm.