Transfer in Reinforcement Learning via Regret Bounds for Learning Agents
This provides theoretical bounds on the benefit of transfer learning for multi-agent reinforcement learning systems, though it is incremental as it builds on existing regret analysis frameworks.
The paper tackles the problem of quantifying the usefulness of transfer in reinforcement learning by analyzing regret bounds in a multi-agent setting, showing that sharing observations reduces total regret by a factor of √ℵ compared to independent learning.
We present an approach for the quantification of the usefulness of transfer in reinforcement learning via regret bounds for a multi-agent setting. Considering a number of $\aleph$ agents operating in the same Markov decision process, however possibly with different reward functions, we consider the regret each agent suffers with respect to an optimal policy maximizing her average reward. We show that when the agents share their observations the total regret of all agents is smaller by a factor of $\sqrt{\aleph}$ compared to the case when each agent has to rely on the information collected by herself. This result demonstrates how considering the regret in multi-agent settings can provide theoretical bounds on the benefit of sharing observations in transfer learning.