COLLIE: Guiding Skill Discovery in Semantically Coherent Latent Space
For researchers in unsupervised skill discovery, COLLIE provides a training-free guidance method that effectively aligns skills with human intent using sparse online feedback, addressing a key limitation of existing guided skill discovery approaches.
COLLIE enables guided skill discovery with minimal human feedback by constructing a semantically coherent latent space from dense unsupervised data, achieving diverse, human-aligned skills and superior downstream performance across state-based and pixel-based tasks.
Unsupervised skill discovery (USD) aims to learn diverse behaviors without reward functions, but often results in task-irrelevant or hazardous behaviors due to uniform exploration. Guided skill discovery (GSD) addresses this issue by incorporating human intent to focus exploration on meaningful regions. However, existing GSD methods typically require training additional guidance models, and rely on pre-defined rules or expert demonstration, which can be ineffective under sparse, online-collected human feedback. To overcome this, we propose COLLIE, a GSD framework that leverages dense unsupervised data to construct a semantically coherent skill latent space. This latent space is well-structured, enabling reliable guidance with sparse online feedback. Moreover, its semantic coherence property enables training-free construction of guidance signals, eliminating the need for additional model training beyond skill learning. Theoretical analysis justifies the effectiveness of our training-free guidance signal, while experiments across diverse state-based and pixel-based tasks show that COLLIE learns diverse, human-aligned skills, avoids hazardous behaviors, and achieves superior downstream performance with minimal human feedback.