13.0LGMay 16, 2022
Loss Landscape Engineering via Data Regulation on PINNsVignesh Gopakumar, Stanislas Pamela, Debasmita Samaddar
Physics-Informed Neural Networks have shown unique utility in parameterising the solution of a well-defined partial differential equation using automatic differentiation and residual losses. Though they provide theoretical guarantees of convergence, in practice the required training regimes tend to be exacting and demanding. Through the course of this paper, we take a deep dive into understanding the loss landscapes associated with a PINN and how that offers some insight as to why PINNs are fundamentally hard to optimise for. We demonstrate how PINNs can be forced to converge better towards the solution, by way of feeding in sparse or coarse data as a regulator. The data regulates and morphs the topology of the loss landscape associated with the PINN to make it easily traversable for the minimiser. Data regulation of PINNs helps ease the optimisation required for convergence by invoking a hybrid unsupervised-supervised training approach, where the labelled data pushes the network towards the vicinity of the solution, and the unlabelled regime fine-tunes it to the solution.
6.3NAApr 1
Finite-Time Convergence Guarantees for Time-Parallel MethodsGiancarlo Antonino Antonucci, Raphael Andreas Hauser, Debasmita Samaddar et al.
Time-parallel algorithms, such as Parareal, are well-understood for linear problems, but their convergence analysis for nonlinear, chaotic systems remains limited. This paper introduces a new theoretical framework for analysing time-decomposition methods as contraction mappings that converge in a finite number of iterations. We derive a finite-time guarantee linking the initial error, convergence rate, and iteration count, defined via a geometric outer--inner-ball condition. We apply this framework to Parareal, deriving explicit estimates for the convergence factor $β$ on nonlinear problems and showing it scales as $\mathcal{O}(h^2)$ when the macroscopic time grid is uniformly refined. Further, we address the failure of standard convergence criteria in chaotic regimes by introducing a proximity function. This chaos-aware criterion weighs solution discontinuities by the system's Lyapunov exponent (or the solver's Lipschitz constant), allowing the algorithm to converge to the correct statistical attractor without enforcing futile pointwise accuracy on divergent trajectories. Numerical experiments on the Logistic, Lorenz, and Lorenz-96 systems demonstrate that this approach decouples the iteration count from the total simulation time. By isolating the intrinsic mathematical bounds from hardware-dependent overheads, we establish that the method is strictly algorithmically scalable.
5.9SPJul 26, 2019
Classification of chaotic time series with deep learningNicolas Boullé, Vassilios Dallas, Yuji Nakatsukasa et al.
We use standard deep neural networks to classify univariate time series generated by discrete and continuous dynamical systems based on their chaotic or non-chaotic behaviour. Our approach to circumvent the lack of precise models for some of the most challenging real-life applications is to train different neural networks on a data set from a dynamical system with a basic or low-dimensional phase space and then use these networks to classify univariate time series of a dynamical system with more intricate or high-dimensional phase space. We illustrate this generalisation approach using the logistic map, the sine-circle map, the Lorenz system, and the Kuramoto--Sivashinsky equation. We observe that a convolutional neural network without batch normalization layers outperforms state-of-the-art neural networks for time series classification and is able to generalise and classify time series as chaotic or not with high accuracy.