CoNES: Convex Natural Evolutionary Strategies
This work addresses the challenge of efficient blackbox optimization for high-dimensional problems, such as in reinforcement learning, with incremental improvements over existing evolutionary strategies.
The authors tackled the problem of optimizing high-dimensional blackbox functions by introducing CoNES, a convex formulation of natural evolutionary strategies, which demonstrated superior performance on benchmark functions and faster convergence on MuJoCo reinforcement learning tasks compared to conventional methods.
We present a novel algorithm -- convex natural evolutionary strategies (CoNES) -- for optimizing high-dimensional blackbox functions by leveraging tools from convex optimization and information geometry. CoNES is formulated as an efficiently-solvable convex program that adapts the evolutionary strategies (ES) gradient estimate to promote rapid convergence. The resulting algorithm is invariant to the parameterization of the belief distribution. Our numerical results demonstrate that CoNES vastly outperforms conventional blackbox optimization methods on a suite of functions used for benchmarking blackbox optimizers. Furthermore, CoNES demonstrates the ability to converge faster than conventional blackbox methods on a selection of OpenAI's MuJoCo reinforcement learning tasks for locomotion.