Martin Hutzenthaler

Martin Hutzenthaler, Arnulf Jentzen, Peter E. Kloeden

On the one hand, the explicit Euler scheme fails to converge strongly to the exact solution of a stochastic differential equation (SDE) with a superlinearly growing and globally one-sided Lipschitz continuous drift coefficient. On the other hand, the implicit Euler scheme is known to converge strongly to the exact solution of such an SDE. Implementations of the implicit Euler scheme, however, require additional computational effort. In this article we therefore propose an explicit and easily implementable numerical method for such an SDE and show that this method converges strongly with the standard order one-half to the exact solution of the SDE. Simulations reveal that this explicit strongly convergent numerical scheme is considerably faster than the implicit Euler scheme.

Martin Hutzenthaler, Arnulf Jentzen

Many stochastic differential equations (SDEs) in the literature have a superlinearly growing nonlinearity in their drift or diffusion coefficient. Unfortunately, moments of the computationally efficient Euler-Maruyama approximation method diverge for these SDEs in finite time. This article develops a general theory based on rare events for studying integrability properties such as moment bounds for discrete-time stochastic processes. Using this approach, we establish moment bounds for fully and partially drift-implicit Euler methods and for a class of new explicit approximation methods which require only a few more arithmetical operations than the Euler-Maruyama method. These moment bounds are then used to prove strong convergence of the proposed schemes. Finally, we illustrate our results for several SDEs from finance, physics, biology and chemistry.

Weinan E, Martin Hutzenthaler, Arnulf Jentzen et al.

We introduce a new family of numerical algorithms for approximating solutions of general high-dimensional semilinear parabolic partial differential equations at single space-time points. The algorithm is obtained through a delicate combination of the Feynman-Kac and the Bismut-Elworthy-Li formulas, and an approximate decomposition of the Picard fixed-point iteration with multilevel accuracy. The algorithm has been tested on a variety of semilinear partial differential equations that arise in physics and finance, with very satisfactory results. Analytical tools needed for the analysis of such algorithms, including a semilinear Feynman-Kac formula, a new class of semi-norms and their recursive inequalities, are also introduced. They allow us to prove for semilinear heat equations with gradient-independent nonlinearity that the computational complexity of the proposed algorithm is bounded by $O(d\,\varepsilon^{-(4+δ)})$ for any $δ\in (0,\infty)$ under suitable assumptions, where $d\in \mathbb{N}$ is the dimensionality of the problem and $\varepsilon\in(0,\infty)$ is the prescribed accuracy.

Martin Hutzenthaler, Arnulf Jentzen, Peter E. Kloeden

The Euler-Maruyama scheme is known to diverge strongly and numerically weakly when applied to nonlinear stochastic differential equations (SDEs) with superlinearly growing and globally one-sided Lipschitz continuous drift coefficients. Classical Monte Carlo simulations do, however, not suffer from this divergence behavior of Euler's method because this divergence behavior happens on rare events. Indeed, for such nonlinear SDEs the classical Monte Carlo Euler method has been shown to converge by exploiting that the Euler approximations diverge only on events whose probabilities decay to zero very rapidly. Significantly more efficient than the classical Monte Carlo Euler method is the recently introduced multilevel Monte Carlo Euler method. The main observation of this article is that this multilevel Monte Carlo Euler method does - in contrast to classical Monte Carlo methods - not converge in general in the case of such nonlinear SDEs. More precisely, we establish divergence of the multilevel Monte Carlo Euler method for a family of SDEs with superlinearly growing and globally one-sided Lipschitz continuous drift coefficients. In particular, the multilevel Monte Carlo Euler method diverges for these nonlinear SDEs on an event that is not at all rare but has probability one. As a consequence for applications, we recommend not to use the multilevel Monte Carlo Euler method for SDEs with superlinearly growing nonlinearities. Instead we propose to combine the multilevel Monte Carlo method with a slightly modified Euler method. More precisely, we show that the multilevel Monte Carlo method combined with a tamed Euler method converges for nonlinear SDEs with globally one-sided Lipschitz continuous drift coefficients and preserves its strikingly higher order convergence rate from the Lipschitz case.

Martin Hutzenthaler, Arnulf Jentzen, Xiaojie Wang

Exponential integrability properties of numerical approximations are a key tool for establishing positive rates of strong and numerically weak convergence for a large class of nonlinear stochastic differential equations. It turns out that well-known numerical approximation processes such as Euler-Maruyama approximations, linear-implicit Euler approximations, and some tamed Euler approximations from the literature rarely preserve exponential integrability properties of the exact solution. The main contribution of this article is to identify a class of stopped increment-tamed Euler approximations which preserve exponential integrability properties of the exact solution under minor additional assumptions on the involved functions.

Martin Hutzenthaler, Arnulf Jentzen, Philippe von Wurstemberger

Parabolic partial differential equations (PDEs) are widely used in the mathematical modeling of natural phenomena and man made complex systems. In particular, parabolic PDEs are a fundamental tool to determine fair prices of financial derivatives in the financial industry. The PDEs appearing in financial engineering applications are often nonlinear and high dimensional since the dimension typically corresponds to the number of considered financial assets. A major issue is that most approximation methods for nonlinear PDEs in the literature suffer under the so-called curse of dimensionality in the sense that the computational effort to compute an approximation with a prescribed accuracy grows exponentially in the dimension of the PDE or in the reciprocal of the prescribed approximation accuracy and nearly all approximation methods have not been shown not to suffer under the curse of dimensionality. Recently, a new class of approximation schemes for semilinear parabolic PDEs, termed full history recursive multilevel Picard (MLP) algorithms, were introduced and it was proven that MLP algorithms do overcome the curse of dimensionality for semilinear heat equations. In this paper we extend those findings to a more general class of semilinear PDEs including as special cases semilinear Black-Scholes equations used for the pricing of financial derivatives with default risks. More specifically, we introduce an MLP algorithm for the approximation of solutions of semilinear Black-Scholes equations and prove that the computational effort of our method grows at most polynomially both in the dimension and the reciprocal of the prescribed approximation accuracy. This is, to the best of our knowledge, the first result showing that the approximation of solutions of semilinear Black-Scholes equations is a polynomially tractable approximation problem.

Matteo Beccari, Martin Hutzenthaler, Arnulf Jentzen et al.

The explicit Euler scheme and similar explicit approximation schemes (such as the Milstein scheme) are known to diverge strongly and numerically weakly in the case of one-dimensional stochastic ordinary differential equations with superlinearly growing nonlinearities. It remained an open question whether such a divergence phenomenon also holds in the case of stochastic partial differential equations with superlinearly growing nonlinearities such as stochastic Allen-Cahn equations. In this work we solve this problem by proving that full-discrete exponential Euler and full-discrete linear-implicit Euler approximations diverge strongly and numerically weakly in the case of stochastic Allen-Cahn equations. This article also contains a short literature overview on existing numerical approximation results for stochastic differential equations with superlinearly growing nonlinearities.

Martin Hutzenthaler, Thomas Kruse

Parabolic partial differential equations (PDEs) and backward stochastic differential equations (BSDEs) have a wide range of applications. In particular, high-dimensional PDEs with gradient-dependent nonlinearities appear often in the state-of-the-art pricing and hedging of financial derivatives. In this article we prove that semilinear heat equations with gradient-dependent nonlinearities can be approximated under suitable assumptions with computational complexity that grows polynomially both in the dimension and the reciprocal of the accuracy.

Martin Hutzenthaler, Arnulf Jentzen, Katharina Pohl et al.

In many numerical simulations stochastic gradient descent (SGD) type optimization methods perform very effectively in the training of deep neural networks (DNNs) but till this day it remains an open problem of research to provide a mathematical convergence analysis which rigorously explains the success of SGD type optimization methods in the training of DNNs. In this work we study SGD type optimization methods in the training of fully-connected feedforward DNNs with rectified linear unit (ReLU) activation. We first establish general regularity properties for the risk functions and their generalized gradient functions appearing in the training of such DNNs and, thereafter, we investigate the plain vanilla SGD optimization method in the training of such DNNs under the assumption that the target function under consideration is a constant function. Specifically, we prove under the assumption that the learning rates (the step sizes of the SGD optimization method) are sufficiently small but not $L^1$-summable and under the assumption that the target function is a constant function that the expectation of the riskof the considered SGD process converges in the training of such DNNs to zero as the number of SGD steps increases to infinity.

Christian Beck, Martin Hutzenthaler, Arnulf Jentzen et al.

It is one of the most challenging problems in applied mathematics to approximatively solve high-dimensional partial differential equations (PDEs). Recently, several deep learning-based approximation algorithms for attacking this problem have been proposed and tested numerically on a number of examples of high-dimensional PDEs. This has given rise to a lively field of research in which deep learning-based methods and related Monte Carlo methods are applied to the approximation of high-dimensional PDEs. In this article we offer an introduction to this field of research by revisiting selected mathematical results related to deep learning approximation methods for PDEs and reviewing the main ideas of their proofs. We also provide a short overview of the recent literature in this area of research.

Sonja Cox, Martin Hutzenthaler, Arnulf Jentzen et al.

We show that if a sequence of piecewise affine linear processes converges in the strong sense with a positive rate to a stochastic process which is strongly Hölder continuous in time, then this sequence converges in the strong sense even with respect to much stronger Hölder norms and the convergence rate is essentially reduced by the Hölder exponent. Our first application hereof establishes pathwise convergence rates for spectral Galerkin approximations of stochastic partial differential equations. Our second application derives strong convergence rates of multilevel Monte Carlo approximations of expectations of Banach space valued stochastic processes.

Weinan E, Martin Hutzenthaler, Arnulf Jentzen et al.

Parabolic partial differential equations (PDEs) and backward stochastic differential equations (BSDEs) are key ingredients in a number of models in physics and financial engineering. In particular, parabolic PDEs and BSDEs are fundamental tools in the state-of-the-art pricing and hedging of financial derivatives. The PDEs and BSDEs appearing in such applications are often high-dimensional and nonlinear. Since explicit solutions of such PDEs and BSDEs are typically not available, it is a very active topic of research to solve such PDEs and BSDEs approximately. In the recent article [E, W., Hutzenthaler, M., Jentzen, A., and Kruse, T. Linear scaling algorithms for solving high-dimensional nonlinear parabolic differential equations. arXiv:1607.03295 (2017)] we proposed a family of approximation methods based on Picard approximations and multilevel Monte Carlo methods and showed under suitable regularity assumptions on the exact solution for semilinear heat equations that the computational complexity is bounded by $O( d \, ε^{-(4+δ)})$ for any $δ\in(0,\infty)$, where $d$ is the dimensionality of the problem and $ε\in(0,\infty)$ is the prescribed accuracy. In this paper, we test the applicability of this algorithm on a variety of $100$-dimensional nonlinear PDEs that arise in physics and finance by means of numerical simulations presenting approximation accuracy against runtime. The simulation results for these 100-dimensional example PDEs are very satisfactory in terms of accuracy and speed. In addition, we also provide a review of other approximation methods for nonlinear PDEs and BSDEs from the literature.

Martin Hutzenthaler, Arnulf Jentzen, Diyora Salimova

This article introduces and analyzes a new explicit, easily implementable, and full discrete accelerated exponential Euler-type approximation scheme for additive space-time white noise driven stochastic partial differential equations (SPDEs) with possibly non-globally monotone nonlinearities such as stochastic Kuramoto-Sivashinsky equations. The main result of this article proves that the proposed approximation scheme converges strongly and numerically weakly to the solution process of such an SPDE. Key ingredients in the proof of our convergence result are a suitable generalized coercivity-type condition, the specific design of the accelerated exponential Euler-type approximation scheme, and an application of Fernique's theorem.

Martin Hutzenthaler, Arnulf Jentzen

Stochastic differential equations are often simulated with the Monte Carlo Euler method. Convergence of this method is well understood in the case of globally Lipschitz continuous coefficients of the stochastic differential equation. The important case of superlinearly growing coefficients, however, has remained an open question. The main difficulty is that numerically weak convergence fails to hold in many cases of superlinearly growing coefficients. In this paper we overcome this difficulty and establish convergence of the Monte Carlo Euler method for a large class of one-dimensional stochastic differential equations whose drift functions have at most polynomial growth.

Martin Hutzenthaler, Arnulf Jentzen, Peter E. Kloeden

The stochastic Euler scheme is known to converge to the exact solution of a stochastic differential equation with globally Lipschitz continuous drift and diffusion coefficient. Recent results extend this convergence to coefficients which grow at most linearly. For superlinearly growing coefficients finite-time convergence in the strong mean square sense remained an open question according to [Higham, Mao & Stuart (2002); Strong convergence of Euler-type methods for nonlinear stochastic differential equations, SIAM J. Numer. Anal. 40, no. 3, 1041-1063]. In this article we answer this question to the negative and prove for a large class of stochastic differential equations with non-globally Lipschitz continuous coefficients that Euler's approximation converges neither in the strong mean square sense nor in the numerically weak sense to the exact solution at a finite time point. Even worse, the difference of the exact solution and of the numerical approximation at a finite time point diverges to infinity in the strong mean square sense and in the numerically weak sense.