Hyperparameter Tuning for Deep Reinforcement Learning Applications
This work addresses the laborious and computationally expensive process of hyperparameter tuning for deep RL, which is crucial for improving the performance and reliability of RL-based controllers in real-world applications, though it is incremental as it builds on existing genetic algorithm methods.
The paper tackles the problem of hyperparameter tuning in deep reinforcement learning by proposing a distributed variable-length genetic algorithm framework, which reduces training episodes and computational cost while improving robustness across various RL applications.
Reinforcement learning (RL) applications, where an agent can simply learn optimal behaviors by interacting with the environment, are quickly gaining tremendous success in a wide variety of applications from controlling simple pendulums to complex data centers. However, setting the right hyperparameters can have a huge impact on the deployed solution performance and reliability in the inference models, produced via RL, used for decision-making. Hyperparameter search itself is a laborious process that requires many iterations and computationally expensive to find the best settings that produce the best neural network architectures. In comparison to other neural network architectures, deep RL has not witnessed much hyperparameter tuning, due to its algorithm complexity and simulation platforms needed. In this paper, we propose a distributed variable-length genetic algorithm framework to systematically tune hyperparameters for various RL applications, improving training time and robustness of the architecture, via evolution. We demonstrate the scalability of our approach on many RL problems (from simple gyms to complex applications) and compared with Bayesian approach. Our results show that with more generations, optimal solutions that require fewer training episodes and are computationally cheap while being more robust for deployment. Our results are imperative to advance deep reinforcement learning controllers for real-world problems.