Ensemble Feature Extraction for Multi-Container Quality-Diversity Algorithms
This addresses a bottleneck in multi-modal optimization for complex reinforcement learning or robotics tasks, though it is incremental over prior automated feature extraction methods.
The paper tackles the problem of Quality-Diversity algorithms' dependency on hand-designed feature descriptors by introducing MC-AURORA, which uses an ensemble of modular auto-encoders to automatically generate multiple diverse feature sets, resulting in more diverse solutions than single-representation approaches.
Quality-Diversity algorithms search for large collections of diverse and high-performing solutions, rather than just for a single solution like typical optimisation methods. They are specially adapted for multi-modal problems that can be solved in many different ways, such as complex reinforcement learning or robotics tasks. However, these approaches are highly dependent on the choice of feature descriptors (FDs) quantifying the similarity in behaviour of the solutions. While FDs usually needs to be hand-designed, recent studies have proposed ways to define them automatically by using feature extraction techniques, such as PCA or Auto-Encoders, to learn a representation of the problem from previously explored solutions. Here, we extend these approaches to more complex problems which cannot be efficiently explored by relying only on a single representation but require instead a set of diverse and complementary representations. We describe MC-AURORA, a Quality-Diversity approach that optimises simultaneously several collections of solutions, each with a different set of FDs, which are, in turn, defined automatically by an ensemble of modular auto-encoders. We show that this approach produces solutions that are more diverse than those produced by single-representation approaches.