Intrinsically-Motivated Reinforcement Learning: A Brief Introduction
This addresses the exploration problem in RL for applications like smart manufacturing and autonomous driving, but it is incremental as it builds on existing intrinsic reward methods.
The paper tackles the exploration-exploitation dilemma in reinforcement learning by introducing intrinsically-motivated RL, proposing a new method based on Rényi state entropy maximization that achieves superior performance with higher efficiency and robustness in simulations.
Reinforcement learning (RL) is one of the three basic paradigms of machine learning. It has demonstrated impressive performance in many complex tasks like Go and StarCraft, which is increasingly involved in smart manufacturing and autonomous driving. However, RL consistently suffers from the exploration-exploitation dilemma. In this paper, we investigated the problem of improving exploration in RL and introduced the intrinsically-motivated RL. In sharp contrast to the classic exploration strategies, intrinsically-motivated RL utilizes the intrinsic learning motivation to provide sustainable exploration incentives. We carefully classified the existing intrinsic reward methods and analyzed their practical drawbacks. Moreover, we proposed a new intrinsic reward method via Rényi state entropy maximization, which overcomes the drawbacks of the preceding methods and provides powerful exploration incentives. Finally, extensive simulation demonstrated that the proposed module achieve superior performance with higher efficiency and robustness.