Michael Kraus

Michael Kraus, Omar Maj

Variational integrators for Lagrangian dynamical systems provide a systematic way to derive geometric numerical methods. These methods preserve a discrete multisymplectic form as well as momenta associated to symmetries of the Lagrangian via Noether's theorem. An inevitable prerequisite for the derivation of variational integrators is the existence of a variational formulation for the considered problem. Even though for a large class of systems this requirement is fulfilled, there are many interesting examples which do not belong to this class, e.g., equations of advection-diffusion type frequently encountered in fluid dynamics or plasma physics. On the other hand, it is always possible to embed an arbitrary dynamical system into a larger Lagrangian system using the method of formal (or adjoint) Lagrangians. We investigate the application of the variational integrator method to formal Lagrangians, and thereby extend the application domain of variational integrators to include potentially all dynamical systems. The theory is supported by physically relevant examples, such as the advection equation and the vorticity equation, and numerically verified. Remarkably, the integrator for the vorticity equation combines Arakawa's discretisation of the Poisson brackets with a symplectic time stepping scheme in a fully covariant way such that the discrete energy is exactly preserved. In the presentation of the results, we try to make the geometric framework of variational integrators accessible to non specialists.

Michael Kraus, Emanuele Tassi, Daniela Grasso

Reduced magnetohydrodynamics is a simplified set of magnetohydrodynamics equations with applications to both fusion and astrophysical plasmas, possessing a noncanonical Hamiltonian structure and consequently a number of conserved functionals. We propose a new discretisation strategy for these equations based on a discrete variational principle applied to a formal Lagrangian. The resulting integrator preserves important quantities like the total energy, magnetic helicity and cross helicity exactly (up to machine precision). As the integrator is free of numerical resistivity, spurious reconnection along current sheets is absent in the ideal case. If effects of electron inertia are added, reconnection of magnetic field lines is allowed, although the resulting model still possesses a noncanonical Hamiltonian structure. After reviewing the conservation laws of the model equations, the adopted variational principle with the related conservation laws are described both at the continuous and discrete level. We verify the favourable properties of the variational integrator in particular with respect to the preservation of the invariants of the models under consideration and compare with results from the literature and those of a pseudo-spectral code.

Michael Kraus, Omar Maj

A variational integrator for ideal magnetohydrodynamics is derived by applying a discrete action principle to a formal Lagrangian. Discrete exterior calculus is used for the discretisation of the field variables in order to preserve their geometrical character. The resulting numerical method is free of numerical resistivity, thus the magnetic field line topology is preserved and unphysical reconnection is absent. In 2D numerical examples we find that important conservation laws like total energy, magnetic helicity and cross helicity are satisfied within machine accuracy.

Eero Hirvijoki, Michael Kraus, Joshua W. Burby

In this paper, we present a new framework for addressing the nonlinear Landau collision operator in terms of particle-in-cell methods. We employ the underlying metriplectic structure of the collision operator and, using a macro particle discretization for the distribution function, we transform the infinite-dimensional system into a finite-dimensional time-continuous metriplectic system for advancing the macro particle weights. Temporal discretization is accomplished using the concept of discrete gradients. The conservation of density, momentum, and energy, as well as the positive semi-definite production of entropy in both the time-continuous and the fully discrete system is demonstrated algebraically. The new algorithm is fully compatible with the existing particle-in-cell Poisson integrators for the Vlasov-Maxwell system.

Eero Hirvijoki, Joshua W. Burby, Michael Kraus

This paper explores energy-, momentum-, density-, and positivity-preserving spatio-temporal discretizations for the nonlinear Landau collision operator. We discuss two approaches, namely direct Galerkin formulations and discretizations of the underlying infinite-dimensional metriplectic structure of the collision integral. The spatial discretizations are chosen to reproduce the time-continuous conservation laws that correspond to Casimir invariants and to guarantee the positivity of the distribution function. Both the direct and the metriplectic discretization are demonstrated to have exact H-theorems and unique, physically exact equilibrium states. Most importantly, the two approaches are shown to coincide, given the chosen Galerkin method. A temporal discretization, preserving all of the mentioned properties, is achieved with so-called discrete gradients. Hence the proposed algorithm successfully translates all properties of the infinite-dimensional time-continuous Landau collision operator to time- and space-discrete sparse-matrix equations suitable for numerical simulation.

Michael Kraus

Recently, an extended version of magnetohydrodynamics that incorporates electron inertia, dubbed inertial magnetohydrodynamics, has been proposed. This model features a noncanonical Hamiltonian formulation with a number of conserved quantities, including the total energy and modified versions of magnetic and cross helicity. In this work, a variational integrator is presented which preserves these conservation laws to machine accuracy. As long as effects due to finite electron mass are neglected, the scheme preserves the magnetic field line topology so that unphysical reconnection is absent. Only when effects of finite electron mass are added, magnetic reconnection takes place. The excellent conservation properties of the method are illustrated by numerical examples in 2D.

Benedikt Brantner, Michael Kraus

Many applications, such as optimization, uncertainty quantification and inverse problems, require repeatedly performing simulations of large-dimensional physical systems for different choices of parameters. This can be prohibitively expensive. In order to save computational cost, one can construct surrogate models by expressing the system in a low-dimensional basis, obtained from training data. This is referred to as model reduction. Past investigations have shown that, when performing model reduction of Hamiltonian systems, it is crucial to preserve the symplectic structure associated with the system in order to ensure long-term numerical stability. Up to this point structure-preserving reductions have largely been limited to linear transformations. We propose a new neural network architecture in the spirit of autoencoders, which are established tools for dimension reduction and feature extraction in data science, to obtain more general mappings. In order to train the network, a non-standard gradient descent approach is applied that leverages the differential-geometric structure emerging from the network design. The new architecture is shown to significantly outperform existing designs in accuracy.

Clémentine Courtès, Emmanuel Franck, Michael Kraus et al.

This work focuses on learning non-canonical Hamiltonian dynamics from data, where long-term predictions require the preservation of structure both in the learned model and in numerical schemes. Previous research focused on either facet, respectively with a potential-based architecture and with degenerate variational integrators, but new issues arise when combining both. In experiments, the learnt model is sometimes numerically unstable due to the gauge dependency of the scheme, rendering long-time simulations impossible. In this paper, we identify this problem and propose two different training strategies to address it, either by directly learning the vector field or by learning a time-discrete dynamics through the scheme. Several numerical test cases assess the ability of the methods to learn complex physical dynamics, like the guiding center from gyrokinetic plasma physics.

Benedikt Brantner, Guillaume de Romemont, Michael Kraus et al.

Two of the many trends in neural network research of the past few years have been (i) the learning of dynamical systems, especially with recurrent neural networks such as long short-term memory networks (LSTMs) and (ii) the introduction of transformer neural networks for natural language processing (NLP) tasks. While some work has been performed on the intersection of these two trends, those efforts were largely limited to using the vanilla transformer directly without adjusting its architecture for the setting of a physical system. In this work we develop a transformer-inspired neural network and use it to learn a dynamical system. We (for the first time) change the activation function of the attention layer to imbue the transformer with structure-preserving properties to improve long-term stability. This is shown to be of great advantage when applying the neural network to learning the trajectory of a rigid body.

Rafael Bischof, Michael Kraus

Physics-Informed Neural Networks (PINN) are algorithms from deep learning leveraging physical laws by including partial differential equations together with a respective set of boundary and initial conditions as penalty terms into their loss function. In this work, we observe the significant role of correctly weighting the combination of multiple competitive loss functions for training PINNs effectively. To this end, we implement and evaluate different methods aiming at balancing the contributions of multiple terms of the PINNs loss function and their gradients. After reviewing of three existing loss scaling approaches (Learning Rate Annealing, GradNorm and SoftAdapt), we propose a novel self-adaptive loss balancing scheme for PINNs named \emph{ReLoBRaLo} (Relative Loss Balancing with Random Lookback). We extensively evaluate the performance of the aforementioned balancing schemes by solving both forward as well as inverse problems on three benchmark PDEs for PINNs: Burgers' equation, Kirchhoff's plate bending equation and Helmholtz's equation. The results show that ReLoBRaLo is able to consistently outperform the baseline of existing scaling methods in terms of accuracy, while also inducing significantly less computational overhead.

Michael Kraus, Irfan Custovic, Walter Kaufmann

Our goal is to transform traditional paper-based instruction into an immersive lesson. This paper presents the conception, workflow and deployment of two MR applications for verification of typical yet geometrically complex structural members: a reinforced concrete corbel and a steel frame. The aim of this research is threefold: (i) to develop and implement the technological feasibility of such applications, (ii) to demonstrate possible use cases in the context of structural engineering lectures and (iii) to evaluate the presented MR examples and the future potential of such MR applications in structural engineering lectures through a survey. The workflow and MR teaching applications were developed with Apple's ARKit. The verification process was reproduced in the MR applications based on conventional exercises taught on paper. Users can navigate independently through the applications and review every single step, including a true-to-scale, spatial representation of the specific component as well as associated verification formulas in the respective step. The applications were used to assess the demand and expectations for immersive teaching techniques among students and instructors through a survey. The participants were asked to test the MR applications on their devices or watch pre-recorded video demonstrations, afterwards perception was elicited through a questionnaire. The results of subsequent data analysis show generally positive judgement of the MR application over the six questioned categories (style, usefulness, ease of use, enjoyment, attitude as well as intention towards using). The statistical analysis revealed (positivity) biases for users with prior XR experience w.r.t. to usage and navigation, while inexperienced users underlined increased enjoyment or excitement with this learning format. The outlook covers identified shortcomings and future developments in this field.

Michael Kraus

We propose and compare several projection methods applied to variational integrators for degenerate Lagrangian systems, whose Lagrangian is of the form $L = \vartheta(q) \cdot \dot{q} - H(q)$ and thus linear in velocities. While previous methods for such systems only work reliably in the case of $\vartheta$ being a linear function of $q$, our methods are long-time stable also for systems where $\vartheta$ is a nonlinear function of $q$. We analyse the properties of the resulting algorithms, in particular with respect to the conservation of energy, momentum maps and symplecticity. In numerical experiments, we verify the favourable properties of the projected integrators and demonstrate their excellent long-time fidelity. In particular, we consider a two-dimensional Lotka-Volterra system, planar point vortices with position-dependent circulation and guiding centre dynamics.

Michael Kraus, Eero Hirvijoki

We present a novel framework for addressing the nonlinear Landau collision integral in terms of finite element and other subspace projection methods. We employ the underlying metriplectic structure of the Landau collision integral and, using a Galerkin discretization for the velocity space, we transform the infinite-dimensional system into a finite-dimensional, time-continuous metriplectic system. Temporal discretization is accomplished using the concept of discrete gradients. The conservation of energy, momentum, and particle densities, as well as the production of entropy is demonstrated algebraically for the fully discrete system. Due to the generality of our approach, the conservation properties and the monotonic behavior of entropy are guaranteed for finite element discretizations in general, independently of the mesh configuration.

Michael Kraus, Katharina Kormann, Philip J. Morrison et al.

We present a novel framework for Finite Element Particle-in-Cell methods based on the discretization of the underlying Hamiltonian structure of the Vlasov-Maxwell system. We derive a semi-discrete Poisson bracket, which retains the defining properties of a bracket, anti-symmetry and the Jacobi identity, as well as conservation of its Casimir invariants, implying that the semi-discrete system is still a Hamiltonian system. In order to obtain a fully discrete Poisson integrator, the semi-discrete bracket is used in conjunction with Hamiltonian splitting methods for integration in time. Techniques from Finite Element Exterior Calculus ensure conservation of the divergence of the magnetic field and Gauss' law as well as stability of the field solver. The resulting methods are gauge invariant, feature exact charge conservation and show excellent long-time energy and momentum behaviour. Due to the generality of our framework, these conservation properties are guaranteed independently of a particular choice of the Finite Element basis, as long as the corresponding Finite Element spaces satisfy certain compatibility conditions.

Michael Kraus

Variational integrators are a special kind of geometric discretisation methods applicable to any system of differential equations that obeys a Lagrangian formulation. In this thesis, variational integrators are developed for several important models of plasma physics: guiding centre dynamics (particle dynamics), the Vlasov-Poisson system (kinetic theory), and ideal magnetohydrodynamics (plasma fluid theory). Special attention is given to physical conservation laws like conservation of energy and momentum. Most systems in plasma physics do not possess a Lagrangian formulation to which the variational integrator methodology is directly applicable. Therefore the theory is extended towards nonvariational differential equations by linking it to Ibragimov's theory of integrating factors and adjoint equations. It allows us to find a Lagrangian for all ordinary and partial differential equations and systems thereof. Consequently, the applicability of variational integrators is extended to a much larger family of systems than envisaged in the original theory. This approach allows for the application of Noether's theorem to analyse the conservation properties of the system, both at the continuous and the discrete level. In numerical examples, the conservation properties of the derived schemes are analysed. In case of guiding centre dynamics, momentum in the toroidal direction of a tokamak is preserved exactly. The particle energy exhibits an error, but the absolute value of this error stays constant during the entire simulation. Therefore numerical dissipation is absent. In case of the kinetic theory, the total number of particles, total linear momentum and total energy are preserved exactly, i.e., up to machine accuracy. In case of magnetohydrodynamics, the total energy, cross helicity and the divergence of the magnetic field are preserved up to machine precision.