Self Paced Gaussian Contextual Reinforcement Learning
This provides a scalable alternative for curriculum generation in continuous and partially observable RL domains, though it is incremental as it builds on existing self-paced methods.
The paper tackles the computational inefficiency of self-paced curriculum learning in reinforcement learning by proposing Self-Paced Gaussian Curriculum Learning (SPGL), which uses a closed-form update rule to reduce overhead while matching or outperforming existing methods in benchmarks like Point Mass and Lunar Lander.
Curriculum learning improves reinforcement learning (RL) efficiency by sequencing tasks from simple to complex. However, many self-paced curriculum methods rely on computationally expensive inner-loop optimizations, limiting their scalability in high-dimensional context spaces. In this paper, we propose Self-Paced Gaussian Curriculum Learning (SPGL), a novel approach that avoids costly numerical procedures by leveraging a closed-form update rule for Gaussian context distributions. SPGL maintains the sample efficiency and adaptability of traditional self-paced methods while substantially reducing computational overhead. We provide theoretical guarantees on convergence and validate our method across several contextual RL benchmarks, including the Point Mass, Lunar Lander, and Ball Catching environments. Experimental results show that SPGL matches or outperforms existing curriculum methods, especially in hidden context scenarios, and achieves more stable context distribution convergence. Our method offers a scalable, principled alternative for curriculum generation in challenging continuous and partially observable domains.