Local Policy Optimization for Trajectory-Centric Reinforcement Learning
This addresses the problem of robotic manipulation tasks by providing a more stable and efficient method, though it appears incremental as it builds on existing trajectory-centric approaches.
The paper tackles the challenge of poor real-world performance in trajectory-centric model-based reinforcement learning due to model inaccuracies by formulating trajectory optimization and local policy synthesis as a single nonlinear programming problem, achieving improved performance under simplifying assumptions.
The goal of this paper is to present a method for simultaneous trajectory and local stabilizing policy optimization to generate local policies for trajectory-centric model-based reinforcement learning (MBRL). This is motivated by the fact that global policy optimization for non-linear systems could be a very challenging problem both algorithmically and numerically. However, a lot of robotic manipulation tasks are trajectory-centric, and thus do not require a global model or policy. Due to inaccuracies in the learned model estimates, an open-loop trajectory optimization process mostly results in very poor performance when used on the real system. Motivated by these problems, we try to formulate the problem of trajectory optimization and local policy synthesis as a single optimization problem. It is then solved simultaneously as an instance of nonlinear programming. We provide some results for analysis as well as achieved performance of the proposed technique under some simplifying assumptions.