Evolving Rewards to Automate Reinforcement Learning
This addresses the tedious hand-tuning of rewards in RL for continuous control tasks, offering an incremental automation solution.
The paper tackles the problem of suboptimal policies in reinforcement learning (RL) for continuous control tasks by automating reward tuning with AutoRL, an evolutionary layer that treats reward tuning as hyperparameter optimization, resulting in improvements over baselines, especially for more complex tasks.
Many continuous control tasks have easily formulated objectives, yet using them directly as a reward in reinforcement learning (RL) leads to suboptimal policies. Therefore, many classical control tasks guide RL training using complex rewards, which require tedious hand-tuning. We automate the reward search with AutoRL, an evolutionary layer over standard RL that treats reward tuning as hyperparameter optimization and trains a population of RL agents to find a reward that maximizes the task objective. AutoRL, evaluated on four Mujoco continuous control tasks over two RL algorithms, shows improvements over baselines, with the the biggest uplift for more complex tasks. The video can be found at: \url{https://youtu.be/svdaOFfQyC8}.