Martin Hutzenthaler

Matteo Beccari, Martin Hutzenthaler, Arnulf Jentzen et al.

The explicit Euler scheme and similar explicit approximation schemes (such as the Milstein scheme) are known to diverge strongly and numerically weakly in the case of one-dimensional stochastic ordinary differential equations with superlinearly growing nonlinearities. It remained an open question whether such a divergence phenomenon also holds in the case of stochastic partial differential equations with superlinearly growing nonlinearities such as stochastic Allen-Cahn equations. In this work we solve this problem by proving that full-discrete exponential Euler and full-discrete linear-implicit Euler approximations diverge strongly and numerically weakly in the case of stochastic Allen-Cahn equations. This article also contains a short literature overview on existing numerical approximation results for stochastic differential equations with superlinearly growing nonlinearities.

Martin Hutzenthaler, Arnulf Jentzen, Katharina Pohl et al.

In many numerical simulations stochastic gradient descent (SGD) type optimization methods perform very effectively in the training of deep neural networks (DNNs) but till this day it remains an open problem of research to provide a mathematical convergence analysis which rigorously explains the success of SGD type optimization methods in the training of DNNs. In this work we study SGD type optimization methods in the training of fully-connected feedforward DNNs with rectified linear unit (ReLU) activation. We first establish general regularity properties for the risk functions and their generalized gradient functions appearing in the training of such DNNs and, thereafter, we investigate the plain vanilla SGD optimization method in the training of such DNNs under the assumption that the target function under consideration is a constant function. Specifically, we prove under the assumption that the learning rates (the step sizes of the SGD optimization method) are sufficiently small but not $L^1$-summable and under the assumption that the target function is a constant function that the expectation of the riskof the considered SGD process converges in the training of such DNNs to zero as the number of SGD steps increases to infinity.

Christian Beck, Martin Hutzenthaler, Arnulf Jentzen et al.

It is one of the most challenging problems in applied mathematics to approximatively solve high-dimensional partial differential equations (PDEs). Recently, several deep learning-based approximation algorithms for attacking this problem have been proposed and tested numerically on a number of examples of high-dimensional PDEs. This has given rise to a lively field of research in which deep learning-based methods and related Monte Carlo methods are applied to the approximation of high-dimensional PDEs. In this article we offer an introduction to this field of research by revisiting selected mathematical results related to deep learning approximation methods for PDEs and reviewing the main ideas of their proofs. We also provide a short overview of the recent literature in this area of research.