Mirror Descent Search and its Acceleration
This work addresses optimization and reinforcement learning challenges for researchers, but it appears incremental as it adapts existing mirror descent techniques to these domains.
The authors tackled the problem of applying mirror descent methods to both black-box optimization and reinforcement learning, proposing Mirror Descent Search (MDS) and its accelerated variants, which they showed converge faster than some state-of-the-art methods in experiments.
In recent years, attention has been focused on the relationship between black-box optimiza- tion problem and reinforcement learning problem. In this research, we propose the Mirror Descent Search (MDS) algorithm which is applicable both for black box optimization prob- lems and reinforcement learning problems. Our method is based on the mirror descent method, which is a general optimization algorithm. The contribution of this research is roughly twofold. We propose two essential algorithms, called MDS and Accelerated Mirror Descent Search (AMDS), and two more approximate algorithms: Gaussian Mirror Descent Search (G-MDS) and Gaussian Accelerated Mirror Descent Search (G-AMDS). This re- search shows that the advanced methods developed in the context of the mirror descent research can be applied to reinforcement learning problem. We also clarify the relationship between an existing reinforcement learning algorithm and our method. With two evaluation experiments, we show our proposed algorithms converge faster than some state-of-the-art methods.