Stephen Roberts

Scott Cameron, Tyron Cameron, Arnu Pretorius et al.

Stochastic differential equations provide a rich class of flexible generative models, capable of describing a wide range of spatio-temporal processes. A host of recent work looks to learn data-representing SDEs, using neural networks and other flexible function approximators. Despite these advances, learning remains computationally expensive due to the sequential nature of SDE integrators. In this work, we propose an importance-sampling estimator for probabilities of observations of SDEs for the purposes of learning. Crucially, the approach we suggest does not rely on such integrators. The proposed method produces lower-variance gradient estimates compared to algorithms based on SDE integrators and has the added advantage of being embarrassingly parallelizable. This facilitates the effective use of large-scale parallel hardware for massive decreases in computation time.

Alexander Camuto, Matthew Willetts, Umut Şimşekli et al.

We study the regularisation induced in neural networks by Gaussian noise injections (GNIs). Though such injections have been extensively studied when applied to data, there have been few studies on understanding the regularising effect they induce when applied to network activations. Here we derive the explicit regulariser of GNIs, obtained by marginalising out the injected noise, and show that it penalises functions with high-frequency components in the Fourier domain; particularly in layers closer to a neural network's output. We show analytically and empirically that such regularisation produces calibrated classifiers with large classification margins.

Jordan Pitt, Christopher Zoppou, Stephen Roberts

We use numerical methods to study the behaviour of the Serre equations in the presence of steep gradients because there are no known analytical solutions for these problems. In keeping with the literature we study a class of initial condition problems that are a smooth approximation to the initial conditions of the dam-break problem. This class of initial condition problems allow us to observe the behaviour of the Serre equations with varying steepness of the initial conditions. The numerical solutions of the Serre equations are justified by demonstrating that as the resolution increases they converge to a solution with little error in conservation of mass, momentum and energy independent of the numerical method. We observe four different structures of the converged numerical solutions depending on the steepness of the initial conditions. Two of these structures were observed in the literature, with the other two not being commonly found in the literature. The numerical solutions are then used to assess how well the analytical solution of the shallow water wave equations captures the mean behaviour of the solution of the Serre equations for the dam-break problem. Lastly the numerical solutions are used to evaluate the usefulness of asymptotic results in the literature to approximate the depth and location of the front of an undular bore.

Christopher Zoppou, Jordan Pitt, Stephen G. Roberts

The nonlinear weakly dispersive Serre equations contain higher-order dispersive terms. This includes a mixed derivative flux term which is difficult to handle numerically. The mix spatial and temporal derivative dispersive term is replaced by a combination of temporal and spatial terms. The Serre equations are re-written so that the system of equations contain homogeneous derivative terms only. The reformulated Serre equations involve the water depth and a new quantity as the conserved variables which are evolved using the finite volume method. The remaining primitive variable, the velocity is obtained by solving a second-order elliptic equation using the finite element method. To avoid the introduction of numerical dispersion that may dominate the physical dispersion, the hybrid scheme has third-order accuracy. Using analytical solutions, laboratory flume data and by simulating the dam-break problem, the proposed scheme is shown to be accurate, simple to implement and stable for a range of problems, including discontinuous flows.