Competitiveness of MAP-Elites against Proximal Policy Optimization on locomotion tasks in deterministic simulations
This work addresses the need for efficient learnable controllers in robotics and automation by demonstrating the competitiveness of evolutionary algorithms, though it is incremental as it focuses on a specific comparison in deterministic simulations.
The paper tackled the problem of comparing modern evolutionary algorithms with deep reinforcement learning for robot locomotion control, showing that MAP-Elites outperforms Proximal Policy Optimization in generating better-performing and more robust controllers for a simulated hexapod robot.
The increasing importance of robots and automation creates a demand for learnable controllers which can be obtained through various approaches such as Evolutionary Algorithms (EAs) or Reinforcement Learning (RL). Unfortunately, these two families of algorithms have mainly developed independently and there are only a few works comparing modern EAs with deep RL algorithms. We show that Multidimensional Archive of Phenotypic Elites (MAP-Elites), which is a modern EA, can deliver better-performing solutions than one of the state-of-the-art RL methods, Proximal Policy Optimization (PPO) in the generation of locomotion controllers for a simulated hexapod robot. Additionally, extensive hyper-parameter tuning shows that MAP-Elites displays greater robustness across seeds and hyper-parameter sets. Generally, this paper demonstrates that EAs combined with modern computational resources display promising characteristics and have the potential to contribute to the state-of-the-art in controller learning.