Fast Moving Natural Evolution Strategy for High-Dimensional Problems
This work addresses optimization challenges for researchers and practitioners dealing with high-dimensional problems, but it is incremental as it builds on an existing state-of-the-art method.
The authors tackled high-dimensional black-box optimization by proposing CR-FM-NES, a variant of natural evolution strategies that uses a restricted covariance matrix representation to achieve linear time and space complexity, resulting in significant speedup over FM-NES and effectiveness over baseline methods like VD-CMA and Sep-CMA in experiments up to 1000 dimensions.
In this work, we propose a new variant of natural evolution strategies (NES) for high-dimensional black-box optimization problems. The proposed method, CR-FM-NES, extends a recently proposed state-of-the-art NES, Fast Moving Natural Evolution Strategy (FM-NES), in order to be applicable in high-dimensional problems. CR-FM-NES builds on an idea using a restricted representation of a covariance matrix instead of using a full covariance matrix, while inheriting an efficiency of FM-NES. The restricted representation of the covariance matrix enables CR-FM-NES to update parameters of a multivariate normal distribution in linear time and space complexity, which can be applied to high-dimensional problems. Our experimental results reveal that CR-FM-NES does not lose the efficiency of FM-NES, and on the contrary, CR-FM-NES has achieved significant speedup compared to FM-NES on some benchmark problems. Furthermore, our numerical experiments using 200, 600, and 1000-dimensional benchmark problems demonstrate that CR-FM-NES is effective over scalable baseline methods, VD-CMA and Sep-CMA.