Efficacy of Modern Neuro-Evolutionary Strategies for Continuous Control Optimization
This work addresses optimization challenges in continuous control for AI and robotics researchers, but it is incremental as it builds on existing neuro-evolutionary methods.
The study analyzed modern neuro-evolutionary strategies for continuous control optimization, finding that OpenAI-ES outperformed or equaled other algorithms on all benchmark problems, and revealed that reward functions optimized for reinforcement learning are not necessarily effective for evolutionary strategies, indicating biases in prior comparisons.
We analyze the efficacy of modern neuro-evolutionary strategies for continuous control optimization. Overall, the results collected on a wide variety of qualitatively different benchmark problems indicate that these methods are generally effective and scale well with respect to the number of parameters and the complexity of the problem. Moreover, they are relatively robust with respect to the setting of hyper-parameters. The comparison of the most promising methods indicates that the OpenAI-ES algorithm outperforms or equals the other algorithms on all considered problems. Moreover, we demonstrate how the reward functions optimized for reinforcement learning methods are not necessarily effective for evolutionary strategies and vice versa. This finding can lead to reconsideration of the relative efficacy of the two classes of algorithm since it implies that the comparisons performed to date are biased toward one or the other class.