MMES: Mixture Model based Evolution Strategy for Large-Scale Optimization
This work addresses the challenge of scaling evolution strategies for large-scale optimization, offering an incremental improvement over CMA-ES with potential benefits for domains requiring high-dimensional global optimization.
The paper tackles the problem of efficient large-scale optimization by proposing MMES, a variant of CMA-ES that uses a mixture model for sampling to reduce time complexity while maintaining competitive performance, achieving scalability to high-dimensional problems and preserving rotational invariance.
This work provides an efficient sampling method for the covariance matrix adaptation evolution strategy (CMA-ES) in large-scale settings. In contract to the Gaussian sampling in CMA-ES, the proposed method generates mutation vectors from a mixture model, which facilitates exploiting the rich variable correlations of the problem landscape within a limited time budget. We analyze the probability distribution of this mixture model and show that it approximates the Gaussian distribution of CMA-ES with a controllable accuracy. We use this sampling method, coupled with a novel method for mutation strength adaptation, to formulate the mixture model based evolution strategy (MMES) -- a CMA-ES variant for large-scale optimization. The numerical simulations show that, while significantly reducing the time complexity of CMA-ES, MMES preserves the rotational invariance, is scalable to high dimensional problems, and is competitive against the state-of-the-arts in performing global optimization.