Lifelong Learning in Multi-Armed Bandits
This work is significant for researchers and practitioners working on multi-armed bandit problems, particularly those concerned with continuous learning and knowledge transfer across sequential tasks.
This paper addresses lifelong learning in multi-armed bandits, aiming to minimize total regret across a series of tasks. The authors propose a 'bandit over bandit' approach with greedy algorithms, demonstrating empirical improvements over prior work in the mortal bandit problem.
Continuously learning and leveraging the knowledge accumulated from prior tasks in order to improve future performance is a long standing machine learning problem. In this paper, we study the problem in the multi-armed bandit framework with the objective to minimize the total regret incurred over a series of tasks. While most bandit algorithms are designed to have a low worst-case regret, we examine here the average regret over bandit instances drawn from some prior distribution which may change over time. We specifically focus on confidence interval tuning of UCB algorithms. We propose a bandit over bandit approach with greedy algorithms and we perform extensive experimental evaluations in both stationary and non-stationary environments. We further apply our solution to the mortal bandit problem, showing empirical improvement over previous work.