Yaxian Xu

Ajay Jasra, Jeremy Heng, Yaxian Xu et al.

We consider a class of finite time horizon nonlinear stochastic optimal control problem, where the control acts additively on the dynamics and the control cost is quadratic. This framework is flexible and has found applications in many domains. Although the optimal control admits a path integral representation for this class of control problems, efficient computation of the associated path integrals remains a challenging Monte Carlo task. The focus of this article is to propose a new Monte Carlo approach that significantly improves upon existing methodology. Our proposed methodology first tackles the issue of exponential growth in variance with the time horizon by casting optimal control estimation as a smoothing problem for a state space model associated with the control problem, and applying smoothing algorithms based on particle Markov chain Monte Carlo. To further reduce computational cost, we then develop a multilevel Monte Carlo method which allows us to obtain an estimator of the optimal control with $\mathcal{O}(ε^2)$ mean squared error with a computational cost of $\mathcal{O}(ε^{-2}\log(ε)^2)$. In contrast, a computational cost of $\mathcal{O}(ε^{-3})$ is required for existing methodology to achieve the same mean squared error. Our approach is illustrated on two numerical examples, which validate our theory.

Ajay Jasra, Kody Law, Yaxian Xu

This paper considers a new approach to using Markov chain Monte Carlo (MCMC) in contexts where one may adopt multilevel (ML) Monte Carlo. The underlying problem is to approximate expectations w.r.t. an underlying probability measure that is associated to a continuum problem, such as a continuous-time stochastic process. It is then assumed that the associated probability measure can only be used (e.g. sampled) under a discretized approximation. In such scenarios, it is known that to achieve a target error, the computational effort can be reduced when using MLMC relative to exact sampling from the most accurate discretized probability. The ideas rely upon introducing hierarchies of the discretizations where less accurate approximations cost less to compute, and using an appropriate collapsing sum expression for the target expectation. If a suitable coupling of the probability measures in the hierarchy is achieved, then a reduction in cost is possible. This article focused on the case where exact sampling from such coupling is not possible. We show that one can construct suitably coupled MCMC kernels when given only access to MCMC kernels which are invariant with respect to each discretized probability measure. We prove, under assumptions, that this coupled MCMC approach in a ML context can reduce the cost to achieve a given error, relative to exact sampling. Our approach is illustrated on a numerical example.