Towards Deep Robot Learning with Optimizer applicable to Non-stationary Problems
This addresses noise-robust optimization for robots learning in real-time, non-stationary environments, representing an incremental improvement over existing methods.
The paper tackles the problem of noise and outliers in real-world robot learning datasets, which are non-stationary, by proposing d-AmsGrad, an improved optimizer that slowly decays the maximum second momentum. The result shows that d-AmsGrad outperforms baselines in robotics problems while maintaining the capability to reach the global optimum.
This paper proposes a new optimizer for deep learning, named d-AmsGrad. In the real-world data, noise and outliers cannot be excluded from dataset to be used for learning robot skills. This problem is especially striking for robots that learn by collecting data in real time, which cannot be sorted manually. Several noise-robust optimizers have therefore been developed to resolve this problem, and one of them, named AmsGrad, which is a variant of Adam optimizer, has a proof of its convergence. However, in practice, it does not improve learning performance in robotics scenarios. This reason is hypothesized that most of robot learning problems are non-stationary, but AmsGrad assumes the maximum second momentum during learning to be stationarily given. In order to adapt to the non-stationary problems, an improved version, which slowly decays the maximum second momentum, is proposed. The proposed optimizer has the same capability of reaching the global optimum as baselines, and its performance outperformed that of the baselines in robotics problems.