Quality Diversity for Robot Learning: Limitations and Future Directions
This work addresses limitations in QD for robot learning by suggesting a more efficient and generalizable approach, though it is incremental as it builds on existing QD and goal-conditioned methods.
The paper argues that current Quality Diversity (QD) methods for robot learning are limited to bounded archives and proposes using a single goal-conditioned policy with a classical planner to achieve O(1) space complexity and generalization to task variants, based on modeling relational graphs between archive cells.
Quality Diversity (QD) has shown great success in discovering high-performing, diverse policies for robot skill learning. While current benchmarks have led to the development of powerful QD methods, we argue that new paradigms must be developed to facilitate open-ended search and generalizability. In particular, many methods focus on learning diverse agents that each move to a different xy position in MAP-Elites-style bounded archives. Here, we show that such tasks can be accomplished with a single, goal-conditioned policy paired with a classical planner, achieving O(1) space complexity w.r.t. the number of policies and generalization to task variants. We hypothesize that this approach is successful because it extracts task-invariant structural knowledge by modeling a relational graph between adjacent cells in the archive. We motivate this view with emerging evidence from computational neuroscience and explore connections between QD and models of cognitive maps in human and other animal brains. We conclude with a discussion exploring the relationships between QD and cognitive maps, and propose future research directions inspired by cognitive maps towards future generalizable algorithms capable of truly open-ended search.