Online Optimization of Curriculum Learning Schedules using Evolutionary Optimization
This work addresses the challenge of optimizing curriculum schedules for reinforcement learning agents, offering an incremental improvement over existing methods.
The paper tackled the problem of automatically generating effective curricula for reinforcement learning agents by combining Curriculum Learning with Rolling Horizon Evolutionary Algorithms, resulting in performance improvements for the final agent, though at the cost of additional evaluation during training.
We propose RHEA CL, which combines Curriculum Learning (CL) with Rolling Horizon Evolutionary Algorithms (RHEA) to automatically produce effective curricula during the training of a reinforcement learning agent. RHEA CL optimizes a population of curricula, using an evolutionary algorithm, and selects the best-performing curriculum as the starting point for the next training epoch. Performance evaluations are conducted after every curriculum step in all environments. We evaluate the algorithm on the \textit{DoorKey} and \textit{DynamicObstacles} environments within the Minigrid framework. It demonstrates adaptability and consistent improvement, particularly in the early stages, while reaching a stable performance later that is capable of outperforming other curriculum learners. In comparison to other curriculum schedules, RHEA CL has been shown to yield performance improvements for the final Reinforcement learning (RL) agent at the cost of additional evaluation during training.