Zlatko Drmač

Matthew J. Colbrook, Zlatko Drmač, Andrew Horning

Koopman operators provide a linear framework for data-driven analyses of nonlinear dynamical systems, but their infinite-dimensional nature presents major computational challenges. In this article, we offer an introductory guide to Koopman learning, emphasizing rigorously convergent data-driven methods for forecasting and spectral analysis. We provide a unified account of error control via residuals in both finite- and infinite-dimensional settings, an elementary proof of convergence for generalized Laplace analysis -- a variant of filtered power iteration that works for operators with continuous spectra and no spectral gaps -- and review state-of-the-art approaches for computing continuous spectra and spectral measures. The goal is to provide both newcomers and experts with a clear, structured overview of reliable data-driven techniques for Koopman spectral analysis.

Ryan Mohr, Maria Fonoberova, Zlatko Drmač et al.

Hierarchical support vector regression (HSVR) models a function from data as a linear combination of SVR models at a range of scales, starting at a coarse scale and moving to finer scales as the hierarchy continues. In the original formulation of HSVR, there were no rules for choosing the depth of the model. In this paper, we observe in a number of models a phase transition in the training error -- the error remains relatively constant as layers are added, until a critical scale is passed, at which point the training error drops close to zero and remains nearly constant for added layers. We introduce a method to predict this critical scale a priori with the prediction based on the support of either a Fourier transform of the data or the Dynamic Mode Decomposition (DMD) spectrum. This allows us to determine the required number of layers prior to training any models.

Iva Manojlović, Maria Fonoberova, Ryan Mohr et al.

We consider the training process of a neural network as a dynamical system acting on the high-dimensional weight space. Each epoch is an application of the map induced by the optimization algorithm and the loss function. Using this induced map, we can apply observables on the weight space and measure their evolution. The evolution of the observables are given by the Koopman operator associated with the induced dynamical system. We use the spectrum and modes of the Koopman operator to realize the above objectives. Our methods can help to, a priori, determine the network depth; determine if we have a bad initialization of the network weights, allowing a restart before training too long; speeding up the training time. Additionally, our methods help enable noise rejection and improve robustness. We show how the Koopman spectrum can be used to determine the number of layers required for the architecture. Additionally, we show how we can elucidate the convergence versus non-convergence of the training process by monitoring the spectrum, in particular, how the existence of eigenvalues clustering around 1 determines when to terminate the learning process. We also show how using Koopman modes we can selectively prune the network to speed up the training procedure. Finally, we show that incorporating loss functions based on negative Sobolev norms can allow for the reconstruction of a multi-scale signal polluted by very large amounts of noise.

Zlatko Drmač, Benjamin Peherstorfer

Loewner rational interpolation provides a versatile tool to learn low-dimensional dynamical-system models from frequency-response measurements. This work investigates the robustness of the Loewner approach to noise. The key finding is that if the measurements are polluted with Gaussian noise, then the error due to noise grows at most linearly with the standard deviation with high probability under certain conditions. The analysis gives insights into making the Loewner approach robust against noise via linear transformations and judicious selections of measurements. Numerical results demonstrate the linear growth of the error on benchmark examples.