Automating Curriculum Learning for Reinforcement Learning using a Skill-Based Bayesian Network
This work addresses the problem of automating curriculum learning for reinforcement learning practitioners, offering a novel method that is incremental in improving training efficiency.
The paper tackles the challenge of automatically generating curricula for reinforcement learning to reduce training time or improve performance, introducing SEBNs (Skill-Environment Bayesian Networks) that model relationships between skills, goals, and environment features to predict policy performance, with results showing that curricula based on SEBN frequently outperform other baselines in discrete gridworld, continuous control, and simulated robotics environments.
A major challenge for reinforcement learning is automatically generating curricula to reduce training time or improve performance in some target task. We introduce SEBNs (Skill-Environment Bayesian Networks) which model a probabilistic relationship between a set of skills, a set of goals that relate to the reward structure, and a set of environment features to predict policy performance on (possibly unseen) tasks. We develop an algorithm that uses the inferred estimates of agent success from SEBN to weigh the possible next tasks by expected improvement. We evaluate the benefit of the resulting curriculum on three environments: a discrete gridworld, continuous control, and simulated robotics. The results show that curricula constructed using SEBN frequently outperform other baselines.