QUOTA: The Quantile Option Architecture for Reinforcement Learning
This addresses exploration challenges for reinforcement learning practitioners, but appears incremental as it builds on distributional RL advances.
The paper tackles the exploration problem in reinforcement learning by proposing the Quantile Option Architecture (QUOTA), which uses quantiles of a value distribution instead of the mean, and demonstrates performance advantages in video games and robot simulators.
In this paper, we propose the Quantile Option Architecture (QUOTA) for exploration based on recent advances in distributional reinforcement learning (RL). In QUOTA, decision making is based on quantiles of a value distribution, not only the mean. QUOTA provides a new dimension for exploration via making use of both optimism and pessimism of a value distribution. We demonstrate the performance advantage of QUOTA in both challenging video games and physical robot simulators.