Unsupervised Discovery of Continuous Skills on a Sphere
This addresses the problem of limited behavioral variety in unsupervised skill discovery for robotics, representing an incremental improvement over prior discrete skill methods.
The paper tackles the limitation of existing unsupervised reinforcement learning methods that learn only a finite number of discrete skills, which restricts behavioral variety, by proposing DISCS, a method that learns continuous skills on a sphere to enable potentially infinite skill diversity. It shows that DISCS learns much more diverse skills than other methods in MuJoCo Ant robot control environments.
Recently, methods for learning diverse skills to generate various behaviors without external rewards have been actively studied as a form of unsupervised reinforcement learning. However, most of the existing methods learn a finite number of discrete skills, and thus the variety of behaviors that can be exhibited with the learned skills is limited. In this paper, we propose a novel method for learning potentially an infinite number of different skills, which is named discovery of continuous skills on a sphere (DISCS). In DISCS, skills are learned by maximizing mutual information between skills and states, and each skill corresponds to a continuous value on a sphere. Because the representations of skills in DISCS are continuous, infinitely diverse skills could be learned. We examine existing methods and DISCS in the MuJoCo Ant robot control environments and show that DISCS can learn much more diverse skills than the other methods.