Sequential Transfer in Multi-armed Bandit with Finite Set of Models
This work addresses the challenge of lifelong learning for agents by enabling incremental knowledge transfer in online settings, which is incremental as it builds on existing bandit methods with a focus on sequential tasks.
The paper tackles the problem of sequential transfer in online learning, specifically within the multi-armed bandit framework, by introducing a novel algorithm based on a method-of-moments approach for task estimation, and derives regret bounds to minimize cumulative regret over a sequence of tasks.
Learning from prior tasks and transferring that experience to improve future performance is critical for building lifelong learning agents. Although results in supervised and reinforcement learning show that transfer may significantly improve the learning performance, most of the literature on transfer is focused on batch learning tasks. In this paper we study the problem of \textit{sequential transfer in online learning}, notably in the multi-armed bandit framework, where the objective is to minimize the cumulative regret over a sequence of tasks by incrementally transferring knowledge from prior tasks. We introduce a novel bandit algorithm based on a method-of-moments approach for the estimation of the possible tasks and derive regret bounds for it.