Combine PPO with NES to Improve Exploration
This work addresses exploration challenges in reinforcement learning for researchers and practitioners, but it is incremental as it builds on existing methods like PPO and NES.
The paper tackled the problem of improving exploration in reinforcement learning by combining neural evolution strategy (NES) with proximal policy optimization (PPO) using parameter transfer and parameter space noise, resulting in demonstrated benefits for PPO in discrete and continuous control tasks.
We introduce two approaches for combining neural evolution strategy (NES) and proximal policy optimization (PPO): parameter transfer and parameter space noise. Parameter transfer is a PPO agent with parameters transferred from a NES agent. Parameter space noise is to directly add noise to the PPO agent`s parameters. We demonstrate that PPO could benefit from both methods through experimental comparison on discrete action environments as well as continuous control tasks