Towards a Principled Learning Rate Adaptation for Natural Evolution Strategies
This work addresses a key parameter tuning issue in NES for optimization practitioners, but it is incremental as it builds on existing NES frameworks.
The paper tackles the problem of setting the learning rate in Natural Evolution Strategies (NES) for black-box continuous optimization by proposing a new adaptation mechanism based on the estimation accuracy of the natural gradient, which speeds up search on easy problems and provides robust performance on difficult ones, showing effectiveness over fixed learning rates in experiments.
Natural Evolution Strategies (NES) is a promising framework for black-box continuous optimization problems. NES optimizes the parameters of a probability distribution based on the estimated natural gradient, and one of the key parameters affecting the performance is the learning rate. We argue that from the viewpoint of the natural gradient method, the learning rate should be determined according to the estimation accuracy of the natural gradient. To do so, we propose a new learning rate adaptation mechanism for NES. The proposed mechanism makes it possible to set a high learning rate for problems that are relatively easy to optimize, which results in speeding up the search. On the other hand, in problems that are difficult to optimize (e.g., multimodal functions), the proposed mechanism makes it possible to set a conservative learning rate when the estimation accuracy of the natural gradient seems to be low, which results in the robust and stable search. The experimental evaluations on unimodal and multimodal functions demonstrate that the proposed mechanism works properly depending on a search situation and is effective over the existing method, i.e., using the fixed learning rate.