Open-Ended Reinforcement Learning with Neural Reward Functions
This work addresses the challenge of open-ended skill learning for robotics and AI agents, offering a novel approach that is incremental compared to existing mutual information-based methods.
The paper tackles the problem of unsupervised skill discovery in reinforcement learning by proposing a method that uses iteratively trained neural reward functions to encourage complex behaviors, achieving results such as front-flips for Half-Cheetah and one-legged running for Humanoid in high-dimensional robotic environments.
Inspired by the great success of unsupervised learning in Computer Vision and Natural Language Processing, the Reinforcement Learning community has recently started to focus more on unsupervised discovery of skills. Most current approaches, like DIAYN or DADS, optimize some form of mutual information objective. We propose a different approach that uses reward functions encoded by neural networks. These are trained iteratively to reward more complex behavior. In high-dimensional robotic environments our approach learns a wide range of interesting skills including front-flips for Half-Cheetah and one-legged running for Humanoid. In the pixel-based Montezuma's Revenge environment our method also works with minimal changes and it learns complex skills that involve interacting with items and visiting diverse locations. The implementation of our approach can be found in this link: https://github.com/amujika/Open-Ended-Reinforcement-Learning-with-Neural-Reward-Functions.