Importance sampling-based approximate optimal planning and control
This addresses scalable planning algorithms for nonlinear systems under constraints, though it appears incremental as it builds on existing approximate optimal control and sampling techniques.
The paper tackles optimal planning and control for nonlinear systems with non-differentiable constraints by proposing a sampling-based method that uses importance sampling to compute optimal weight parameters for policy approximation, demonstrating correctness and efficiency through numerical experiments on linear systems with nonlinear costs and Dubins car motion planning.
In this paper, we propose a sampling-based planning and optimal control method of nonlinear systems under non-differentiable constraints. Motivated by developing scalable planning algorithms, we consider the optimal motion plan to be a feedback controller that can be approximated by a weighted sum of given bases. Given this approximate optimal control formulation, our main contribution is to introduce importance sampling, specifically, model-reference adaptive search algorithm, to iteratively compute the optimal weight parameters, i.e., the weights corresponding to the optimal policy function approximation given chosen bases. The key idea is to perform the search by iteratively estimating a parametrized distribution which converges to a Dirac's Delta that infinitely peaks on the global optimal weights. Then, using this direct policy search, we incorporated trajectory-based verification to ensure that, for a class of nonlinear systems, the obtained policy is not only optimal but robust to bounded disturbances. The correctness and efficiency of the methods are demonstrated through numerical experiments including linear systems with a nonlinear cost function and motion planning for a Dubins car.