Path Integral Control by Reproducing Kernel Hilbert Space Embedding
This work addresses sample efficiency in stochastic optimal control, enabling more effective use of data for tasks like robotics or autonomous systems, though it appears incremental as it builds on existing sample-based approaches.
The authors tackled the problem of stochastic optimal control by embedding path integral control into reproducing kernel Hilbert spaces, resulting in a model-free, non-parametric method that allows sample re-use across tasks and demonstrates improved sample efficiency in numerical tests.
We present an embedding of stochastic optimal control problems, of the so called path integral form, into reproducing kernel Hilbert spaces. Using consistent, sample based estimates of the embedding leads to a model free, non-parametric approach for calculation of an approximate solution to the control problem. This formulation admits a decomposition of the problem into an invariant and task dependent component. Consequently, we make much more efficient use of the sample data compared to previous sample based approaches in this domain, e.g., by allowing sample re-use across tasks. Numerical examples on test problems, which illustrate the sample efficiency, are provided.