Direct Policy Optimization using Deterministic Sampling and Collocation
This work addresses motion planning for robotics, offering a method to handle stochasticity and constraints, but it appears incremental as it builds on existing techniques like direct optimization and sampling.
The paper tackles stochastic optimal-control problems by combining direct trajectory optimization, deterministic sampling, and policy optimization, demonstrating exact recovery of LQR policies for linear cases and robust performance on nonlinear robotic systems with control limits and obstacle avoidance.
We present an approach for approximately solving discrete-time stochastic optimal-control problems by combining direct trajectory optimization, deterministic sampling, and policy optimization. Our feedback motion-planning algorithm uses a quasi-Newton method to simultaneously optimize a reference trajectory, a set of deterministically chosen sample trajectories, and a parameterized policy. We demonstrate that this approach exactly recovers LQR policies in the case of linear dynamics, quadratic objective, and Gaussian disturbances. We also demonstrate the algorithm on several nonlinear, underactuated robotic systems to highlight its performance and ability to handle control limits, safely avoid obstacles, and generate robust plans in the presence of unmodeled dynamics.