Neural Architecture Evolution in Deep Reinforcement Learning for Continuous Control
This addresses the need for automated neural architecture design in reinforcement learning, offering a more efficient alternative to manual tuning or expensive search methods.
The paper tackles the problem of handcrafted neural network architectures in deep reinforcement learning for continuous control by proposing an actor-critic neuroevolution algorithm that automatically finds strong topologies with minimal environmental interaction overhead. Experiments on five benchmarks show it often outperforms baselines and achieves sample-efficient architecture discovery.
Current Deep Reinforcement Learning algorithms still heavily rely on handcrafted neural network architectures. We propose a novel approach to automatically find strong topologies for continuous control tasks while only adding a minor overhead in terms of interactions in the environment. To achieve this, we combine Neuroevolution techniques with off-policy training and propose a novel architecture mutation operator. Experiments on five continuous control benchmarks show that the proposed Actor-Critic Neuroevolution algorithm often outperforms the strong Actor-Critic baseline and is capable of automatically finding topologies in a sample-efficient manner which would otherwise have to be found by expensive architecture search.