Convexity of a stochastic control functional related to importance sampling of Itô diffusions
For researchers in rare event simulation, this provides theoretical justification for using convex optimization methods to solve high-dimensional control problems, though the result is incremental as it extends existing theory.
This paper proves strict convexity of a stochastic control functional for importance sampling of Itô diffusions, enabling dimension-robust solutions via convex optimization and Monte Carlo. The result is applied to analyze a gradient descent algorithm for rare event simulation.
We consider the problem of rare event importance sampling, where the random variable of interest is a path functional of an Itô diffusion computed up to the first exit from a $d$-dimensional bounded domain. Dupuis and Wang (\textit{Ann. Appl. Probab.}, 15 (2005), pp. 1-38) studied the importance sampling problem by formulating it as a stochastic optimal control problem, where the value function is related to the conditional cumulant generating function of the random variable. In this paper, we show that the sufficient conditions for the value function to be twice-differentiable and $α$-uniformly Hölder continuous on the closure of the domain are also sufficient conditions for positive definiteness of the second variation of the control functional on the space of differentiable, $α$-uniformly Hölder continuous $\mathbb{R}^d$-valued feedback controls. We derive an expression for the second variation using Kazamaki's sufficient condition for $L^q$-boundedness of exponential martingales, and using Fredholm theory to prove the finiteness of the moment generating function of the first exit time over any bounded interval containing the origin. The strict convexity result suggests that one may be able to solve the corresponding Hamilton-Jacobi-Bellman boundary value problem in a dimension-robust way, by combining convex optimisation and Monte Carlo methods. We apply the result to analyse a gradient descent algorithm proposed by Hartmann and Schütte (\textit{J. Stat. Mech. Theor. Exp.} (2012), P11004) for efficient rare event simulation.