Goal Exploration via Adaptive Skill Distribution for Goal-Conditioned Reinforcement Learning
This addresses exploration challenges in GCRL for tasks with long horizons and sparse rewards, offering an incremental improvement over existing methods.
The paper tackles exploration inefficiency in goal-conditioned reinforcement learning by introducing GEASD, a framework that uses an adaptive skill distribution to capture environmental patterns, resulting in marked improvements in exploration efficiency and robust generalization to unseen tasks.
Exploration efficiency poses a significant challenge in goal-conditioned reinforcement learning (GCRL) tasks, particularly those with long horizons and sparse rewards. A primary limitation to exploration efficiency is the agent's inability to leverage environmental structural patterns. In this study, we introduce a novel framework, GEASD, designed to capture these patterns through an adaptive skill distribution during the learning process. This distribution optimizes the local entropy of achieved goals within a contextual horizon, enhancing goal-spreading behaviors and facilitating deep exploration in states containing familiar structural patterns. Our experiments reveal marked improvements in exploration efficiency using the adaptive skill distribution compared to a uniform skill distribution. Additionally, the learned skill distribution demonstrates robust generalization capabilities, achieving substantial exploration progress in unseen tasks containing similar local structures.