André Schlichting

André Schlichting, Christian Seis

This paper is concerned with the numerical analysis of the explicit upwind finite volume scheme for numerically solving continuity equations. We are interested in the case where the advecting velocity field has spatial Sobolev regularity and initial data are merely integrable. We estimate the error between approximate solutions constructed by the upwind scheme and distributional solutions of the continuous problem in a Kantorovich-Rubinstein distance, which was recently used for stability estimates for the continuity equation by Seis [23]. Restricted to Cartesian meshes, our estimate shows that the rate of weak convergence is at least of order $1/2$ in the mesh size. The proof relies on a probabilistic interpretation of the upwind scheme Delarue and Lagoutière [9]. We complement the weak convergence result with an example that illustrates that for rough initial data no rates can be expected in strong norms. The same example suggests that the weak order $1/2$ rate is optimal.

André Schlichting, Christian Seis

We study the implicit upwind finite volume scheme for numerically approximating the linear continuity equation in the low regularity DiPerna-Lions setting. That is, we are concerned with advecting velocity fields that are spatially Sobolev regular and data that are merely integrable. We prove that on unstructured regular meshes the rate of convergence of approximate solutions generated by the upwind scheme towards the unique distributional solution of the continuous model is at least 1/2. The numerical error is estimated in terms of logarithmic Kantorovich-Rubinstein distances and provides thus a bound on the rate of weak convergence.

Anna Shalova, André Schlichting, Mark Peletier

We study the limiting dynamics of a large class of noisy gradient descent systems in the overparameterized regime. In this regime the set of global minimizers of the loss is large, and when initialized in a neighbourhood of this zero-loss set a noisy gradient descent algorithm slowly evolves along this set. In some cases this slow evolution has been related to better generalisation properties. We characterize this evolution for the broad class of noisy gradient descent systems in the limit of small step size. Our results show that the structure of the noise affects not just the form of the limiting process, but also the time scale at which the evolution takes place. We apply the theory to Dropout, label noise and classical SGD (minibatching) noise, and show that these evolve on different two time scales. Classical SGD even yields a trivial evolution on both time scales, implying that additional noise is required for regularization. The results are inspired by the training of neural networks, but the theorems apply to noisy gradient descent of any loss that has a non-trivial zero-loss set.