Self-Paced Absolute Learning Progress as a Regularized Approach to Curriculum Learning
This work addresses efficiency issues in reinforcement learning for researchers and practitioners, but it is incremental as it builds on existing ALP methods with a regularization improvement.
The paper tackled the problem of wasted computation in curriculum reinforcement learning due to repeating learned behaviors, by introducing Self-Paced Absolute Learning Progress (SPALP) as a regularization method. The result showed that SPALP achieved comparable performance to the original ALP in three environments and reached it quicker in two of them.
The usability of Reinforcement Learning is restricted by the large computation times it requires. Curriculum Reinforcement Learning speeds up learning by defining a helpful order in which an agent encounters tasks, i.e. from simple to hard. Curricula based on Absolute Learning Progress (ALP) have proven successful in different environments, but waste computation on repeating already learned behaviour in new tasks. We solve this problem by introducing a new regularization method based on Self-Paced (Deep) Learning, called Self-Paced Absolute Learning Progress (SPALP). We evaluate our method in three different environments. Our method achieves performance comparable to original ALP in all cases, and reaches it quicker than ALP in two of them. We illustrate possibilities to further improve the efficiency and performance of SPALP.