Active Reinforcement Learning over MDPs
This work addresses generalization efficiency for reinforcement learning practitioners, but it is incremental as it builds on existing methods like Proximal Policy Optimization with instance selection.
The paper tackles the problem of generalization efficiency in reinforcement learning by proposing an Active Reinforcement Learning framework that selects valuable training instances to reduce resource usage, achieving improved generalization efficiency compared to unselected and unbiased selection methods.
The past decade has seen the rapid development of Reinforcement Learning, which acquires impressive performance with numerous training resources. However, one of the greatest challenges in RL is generalization efficiency (i.e., generalization performance in a unit time). This paper proposes a framework of Active Reinforcement Learning (ARL) over MDPs to improve generalization efficiency in a limited resource by instance selection. Given a number of instances, the algorithm chooses out valuable instances as training sets while training the policy, thereby costing fewer resources. Unlike existing approaches, we attempt to actively select and use training data rather than train on all the given data, thereby costing fewer resources. Furthermore, we introduce a general instance evaluation metrics and selection mechanism into the framework. Experiments results reveal that the proposed framework with Proximal Policy Optimization as policy optimizer can effectively improve generalization efficiency than unselect-ed and unbiased selected methods.