Towards Autonomous Reinforcement Learning: Automatic Setting of Hyper-parameters using Bayesian Optimization
This work addresses the need for automatic hyper-parameter tuning in reinforcement learning to enhance performance regardless of user expertise, though it is incremental as it builds on existing Bayesian optimization methods.
The paper tackles the problem of automatically setting hyper-parameters in reinforcement learning to improve agent efficiency in uncertain environments, proposing a framework that integrates Bayesian optimization with Gaussian process regression and a bandits-based approach, demonstrated on a gridworld example with SARSA.
With the increase of machine learning usage by industries and scientific communities in a variety of tasks such as text mining, image recognition and self-driving cars, automatic setting of hyper-parameter in learning algorithms is a key factor for achieving satisfactory performance regardless of user expertise in the inner workings of the techniques and methodologies. In particular, for a reinforcement learning algorithm, the efficiency of an agent learning a control policy in an uncertain environment is heavily dependent on the hyper-parameters used to balance exploration with exploitation. In this work, an autonomous learning framework that integrates Bayesian optimization with Gaussian process regression to optimize the hyper-parameters of a reinforcement learning algorithm, is proposed. Also, a bandits-based approach to achieve a balance between computational costs and decreasing uncertainty about the Q-values, is presented. A gridworld example is used to highlight how hyper-parameter configurations of a learning algorithm (SARSA) are iteratively improved based on two performance functions.