Online Transfer Learning in Reinforcement Learning Domains
This work addresses transfer learning for reinforcement learning agents, but it is incremental as it re-characterizes existing methods.
The paper tackles the problem of online transfer learning in reinforcement learning by proposing a framework that generalizes existing methods and proves convergence for Q-learning and Sarsa with tabular and linear function approximations, showing asymptotic performance is not harmed, with empirical validation.
This paper proposes an online transfer framework to capture the interaction among agents and shows that current transfer learning in reinforcement learning is a special case of online transfer. Furthermore, this paper re-characterizes existing agents-teaching-agents methods as online transfer and analyze one such teaching method in three ways. First, the convergence of Q-learning and Sarsa with tabular representation with a finite budget is proven. Second, the convergence of Q-learning and Sarsa with linear function approximation is established. Third, the we show the asymptotic performance cannot be hurt through teaching. Additionally, all theoretical results are empirically validated.