Quality Diversity for Multi-task Optimization
This addresses multi-task optimization for robotics and AI, but it is incremental as it extends an existing algorithm to handle task-dependent fitness.
The paper tackles the problem of multi-task optimization where fitness functions depend on tasks, proposing Multi-task MAP-Elites to extend Quality Diversity algorithms. Results show it outperforms separate CMA-ES optimization on simulated robotic tasks, such as a 10-dimensional arm with 5000 tasks and a 36-dimensional robot with 2000 tasks.
Quality Diversity (QD) algorithms are a recent family of optimization algorithms that search for a large set of diverse but high-performing solutions. In some specific situations, they can solve multiple tasks at once. For instance, they can find the joint positions required for a robotic arm to reach a set of points, which can also be solved by running a classic optimizer for each target point. However, they cannot solve multiple tasks when the fitness needs to be evaluated independently for each task (e.g., optimizing policies to grasp many different objects). In this paper, we propose an extension of the MAP-Elites algorithm, called Multi-task MAP-Elites, that solves multiple tasks when the fitness function depends on the task. We evaluate it on a simulated parameterized planar arm (10-dimensional search space; 5000 tasks) and on a simulated 6-legged robot with legs of different lengths (36-dimensional search space; 2000 tasks). The results show that in both cases our algorithm outperforms the optimization of each task separately with the CMA-ES algorithm.