Scalable Coordinated Exploration in Concurrent Reinforcement Learning
This addresses the challenge of scalable exploration in multi-agent RL, which is incremental as it builds on existing methods like seed sampling and randomized value functions.
The paper tackles the problem of efficient coordinated exploration for a team of reinforcement learning agents in a common environment, demonstrating that the approach learns quickly with far fewer agents than alternative schemes in a higher-dimensional problem.
We consider a team of reinforcement learning agents that concurrently operate in a common environment, and we develop an approach to efficient coordinated exploration that is suitable for problems of practical scale. Our approach builds on seed sampling (Dimakopoulou and Van Roy, 2018) and randomized value function learning (Osband et al., 2016). We demonstrate that, for simple tabular contexts, the approach is competitive with previously proposed tabular model learning methods (Dimakopoulou and Van Roy, 2018). With a higher-dimensional problem and a neural network value function representation, the approach learns quickly with far fewer agents than alternative exploration schemes.