Nonparametric Infinite Horizon Kullback-Leibler Stochastic Control
For researchers in stochastic control, this work provides nonparametric frameworks that improve upon parametric methods, but the contribution is incremental as it adapts existing techniques (GP, Nyström) to KL control.
The paper presents two nonparametric approaches to Kullback-Leibler (KL) control using Gaussian processes and Nyström approximation, achieving accurate function approximation and efficient online operations. Numerical results demonstrate effectiveness, though no specific performance numbers are given.
We present two nonparametric approaches to Kullback-Leibler (KL) control, or linearly-solvable Markov decision problem (LMDP) based on Gaussian processes (GP) and Nyström approximation. Compared to recently developed parametric methods, the proposed data-driven frameworks feature accurate function approximation and efficient on-line operations. Theoretically, we derive the mathematical connection of KL control based on dynamic programming with earlier work in control theory which relies on information theoretic dualities for the infinite time horizon case. Algorithmically, we give explicit optimal control policies in nonparametric forms, and propose on-line update schemes with budgeted computational costs. Numerical results demonstrate the effectiveness and usefulness of the proposed frameworks.