Quality-Diversity Optimization: a novel branch of stochastic optimization
This work provides a foundational introduction to Quality-Diversity optimization for researchers and practitioners in evolutionary computation, deep learning, robotics, and reinforcement learning.
This paper introduces Quality-Diversity (QD) optimization, a new branch of stochastic optimization that aims to illuminate the entire search space by finding a diverse set of high-performing solutions, rather than just a single optimum. It differs from multimodal optimization by operating in the behavioral space and attempting to fill the entire behavior space.
Traditional optimization algorithms search for a single global optimum that maximizes (or minimizes) the objective function. Multimodal optimization algorithms search for the highest peaks in the search space that can be more than one. Quality-Diversity algorithms are a recent addition to the evolutionary computation toolbox that do not only search for a single set of local optima, but instead try to illuminate the search space. In effect, they provide a holistic view of how high-performing solutions are distributed throughout a search space. The main differences with multimodal optimization algorithms are that (1) Quality-Diversity typically works in the behavioral space (or feature space), and not in the genotypic (or parameter) space, and (2) Quality-Diversity attempts to fill the whole behavior space, even if the niche is not a peak in the fitness landscape. In this chapter, we provide a gentle introduction to Quality-Diversity optimization, discuss the main representative algorithms, and the main current topics under consideration in the community. Throughout the chapter, we also discuss several successful applications of Quality-Diversity algorithms, including deep learning, robotics, and reinforcement learning.