Brendan Keith

Andrew Gillette, Brendan Keith, Socratis Petrides

In this work, we revisit the marking decisions made in the standard adaptive finite element method (AFEM). Experience shows that a naïve marking policy leads to inefficient use of computational resources for adaptive mesh refinement (AMR). Consequently, using AFEM in practice often involves ad-hoc or time-consuming offline parameter tuning to set appropriate parameters for the marking subroutine. To address these practical concerns, we recast AMR as a Markov decision process in which refinement parameters can be selected on-the-fly at run time, without the need for pre-tuning by expert users. In this new paradigm, the refinement parameters are also chosen adaptively via a marking policy that can be optimized using methods from reinforcement learning. We use the Poisson equation to demonstrate our techniques on $h$- and $hp$-refinement benchmark problems, and our experiments suggest that superior marking policies remain undiscovered for many classical AFEM applications. Furthermore, an unexpected observation from this work is that marking policies trained on one family of PDEs are sometimes robust enough to perform well on problems far outside the training family. For illustration, we show that a simple $hp$-refinement policy trained on 2D domains with only a single re-entrant corner can be deployed on far more complicated 2D domains, and even 3D domains, without significant performance loss. For reproduction and broader adoption, we accompany this work with an open-source implementation of our methods.

Brendan Keith, Federico Fuentes, Leszek Demkowicz

The flexibility of the DPG methodology is exposed by solving the linear elasticity equations under different variational formulations, including some with non-symmetric functional settings (different infinite-dimensional trial and test spaces). The family of formulations presented are proved to be mutually ill or well-posed when using traditional energy spaces on the whole domain. Moreover, they are shown to remain well-posed when using broken energy spaces and interface variables. Four variational formulations are solved in 3D using the DPG methodology. Numerical evidence is given for both smooth and singular solutions and the expected convergence rates are observed.

Brendan Keith, Socratis Petrides, Federico Fuentes et al.

A finite element methodology for large classes of variational boundary value problems is defined which involves discretizing two linear operators: (1) the differential operator defining the spatial boundary value problem; and (2) a Riesz map on the test space. The resulting linear system is overdetermined. Two different approaches for solving the system are suggested (although others are discussed): (1) solving the associated normal equation with linear solvers for symmetric positive-definite systems (e.g. Cholesky factorization); and (2) solving the overdetermined system with orthogonalization algorithms (e.g. QR factorization). The finite element assembly algorithm for each of these approaches is described in detail. The normal equation approach is usually faster for direct solvers and requires less storage. The second approach reduces the condition number of the system by a power of two and is less sensitive to round-off error. The rectangular stiffness matrix of second approach is demonstrated to have condition number $\mathcal{O}(h^{-1})$ for a variety of formulations of Poisson's equation. The stiffness matrix from the normal equation approach is demonstrated to be related to the monolithic stiffness matrices of least-squares finite element methods and it is proved that the two are identical in some cases. An example with Poisson's equation indicates that the solutions of these two different linear systems can be nearly indistinguishable (if round-off error is not an issue) and rapidly converge to each other. The orthogonalization approach is suggested to be beneficial for problems which induce poorly conditioned linear systems. Experiments with Poisson's equation in single-precision arithmetic as well as the linear acoustics problem near resonance in double-precision arithmetic verify this conclusion.

Ali Vaziri Astaneh, Brendan Keith, Leszek Demkowicz

In this article, several discontinuous Petrov-Galerkin (DPG) methods with perfectly matched layers (PMLs) are derived along with their quasi-optimal graph test norms. Ultimately, two different complex coordinate stretching strategies are considered in these derivations. Unlike with classical formulations used by Bubnov-Galerkin methods, with so-called ultraweak variational formulations, these two strategies in fact deliver different formulations in the PML region. One of the strategies, which is argued to be more physically natural, is employed for numerically solving two- and three-dimensional time-harmonic acoustic, elastic, and electromagnetic wave propagation problems, defined in unbounded domains. Through these numerical experiments, efficacy of the new DPG methods with PMLs is verified.

Daniel Drzisga, Brendan Keith, Barbara Wohlmuth

We give the first mathematically rigorous analysis of an emerging approach to finite element analysis (see, e.g., Bauer et al. [Appl. Numer. Math., 2017]), which we hereby refer to as the surrogate matrix methodology. This methodology is based on the piece-wise smooth approximation of the matrices involved in a standard finite element discretization. In particular, it relies on the projection of smooth so-called stencil functions onto high-order polynomial subspaces. The performance advantage of the surrogate matrix methodology is seen in constructions where each stencil function uniquely determines the values of a significant collection of matrix entries. Such constructions are shown to be widely achievable through the use of locally-structured meshes. Therefore, this methodology can be applied to a wide variety of physically meaningful problems, including nonlinear problems and problems with curvilinear geometries. Rigorous a priori error analysis certifies the convergence of a novel surrogate method for the variable coefficient Poisson equation. The flexibility of the methodology is also demonstrated through the construction of novel methods for linear elasticity and nonlinear diffusion problems. In numerous numerical experiments, we demonstrate the efficacy of these new methods in a matrix-free environment with geometric multigrid solvers. In our experiments, up to a twenty-fold decrease in computation time is witnessed over the classical method with an otherwise identical implementation.

Brendan Keith, Leszek Demkowicz, Jay Gopalakrishnan

We introduce a cousin of the DPG method - the DPG* method - discuss their relationship and compare the two methods through numerical experiments.

Jiachen Yang, Ketan Mittal, Tarik Dzanic et al.

Adaptive mesh refinement (AMR) is necessary for efficient finite element simulations of complex physical phenomenon, as it allocates limited computational budget based on the need for higher or lower resolution, which varies over space and time. We present a novel formulation of AMR as a fully-cooperative Markov game, in which each element is an independent agent who makes refinement and de-refinement choices based on local information. We design a novel deep multi-agent reinforcement learning (MARL) algorithm called Value Decomposition Graph Network (VDGN), which solves the two core challenges that AMR poses for MARL: posthumous credit assignment due to agent creation and deletion, and unstructured observations due to the diversity of mesh geometries. For the first time, we show that MARL enables anticipatory refinement of regions that will encounter complex features at future times, thereby unlocking entirely new regions of the error-cost objective landscape that are inaccessible by traditional methods based on local error estimators. Comprehensive experiments show that VDGN policies significantly outperform error threshold-based policies in global error and cost metrics. We show that learned policies generalize to test problems with physical features, mesh geometries, and longer simulation times that were not seen in training. We also extend VDGN with multi-objective optimization capabilities to find the Pareto front of the tradeoff between cost and error.

Peter Gangl, Brendan Keith, Dohyun Kim et al.

We introduce an efficient and scalable method for density-based multi-material topology optimization, integrating classical mirror descent techniques with point-wise polytopal design constraints. Such constraints arise naturally in this class of problems, wherein the vertices of convex polytopes correspond to distinct design states, only one of which should be occupied at each point in space. The framework generates a descending sequence of iterates by penalizing the design space around the previous iterate with a generalized distance function tailored to the convex geometry of the $n$-dimensional polytope. This distance function, called a Bregman divergence, smooths the optimization landscape, ensuring that each iterate strictly satisfies the point-wise constraints. Subsequently, global constraints (e.g., bounds on the structural mass) can be enforced easily by solving a small, finite-dimensional dual problem. The resulting method is simple to implement and demonstrates robustness and efficiency when combined with an Armijo-type line search algorithm. We validate the method in structural design problems involving the optimal arrangement of both isotropic and anisotropic materials, as well as magnetic flux optimization in electric motors.

Brendan Keith, Frédéric Marazzato

We resolve a longstanding open problem in the computational modeling of nonlinear plates by introducing a numerical method that exactly enforces the isometry constraint, namely, that the first fundamental form of the mid-surface coincides with the identity tensor. Several numerical methods have been proposed to approximate solutions of such manifold-constrained variational problems using gradient flows with tangent space updates. However, this class of methods presents two main challenges. First, a preprocessing step is required to enforce the boundary conditions and generate an initial guess sufficiently close to an isometry. Second, each step of the gradient flow typically increases the isometry defect. We adopt an alternative approach based on the proximal Galerkin framework, originally introduced for variational problems with convex inequality constraints. The resulting method preserves the geometric structure of the feasible set and yields an efficient algorithm in which each iterate is an exact isometry at the barycenter of every mesh cell. In contrast to existing methods, no preprocessing step is required, enabling broader applicability of this important category of mathematical models. Numerical experiments on standard benchmarks demonstrate that the method converges to a prescribed error tolerance in an asymptotically mesh-independent number of iterations and requires substantially fewer iterations than previous methods, even on coarse meshes.

Hilarie Sit, Brendan Keith, Karianne Bergen

We propose a prototype-based approach for improving explainability of softmax classifiers that provides an understandable prediction confidence, generated through stochastic sampling of prototypes, and demonstrates potential for out of distribution detection (OOD). By modifying the model architecture and training to make predictions using similarities to any set of class examples from the training dataset, we acquire the ability to sample for prototypical examples that contributed to the prediction, which provide an instance-based explanation for the model's decision. Furthermore, by learning relationships between images from the training dataset through relative distances within the model's latent space, we obtain a metric for uncertainty that is better able to detect out of distribution data than softmax confidence.

Miguel Castillón, Biswajit Khara, Jørgen S. Dokken et al.

The phase-field method has emerged as a powerful tool for simulating fracture mechanics, yet it presents significant numerical challenges, particularly regarding the enforcement of physical constraints such as irreversibility and boundedness of the phase-field variable. This work proposes the proximal Galerkin (PG) methodology as a robust and efficient framework for solving phase-field fracture problems. By reformulating the inequality-constrained optimization problem into a sequence of saddle-point problems involving latent variables, the PG method rigorously enforces the physical bounds of the phase-field variable and naturally handles the irreversibility condition. This approach is directly applicable to both static and dynamic phase-field fracture problems. The numerical results demonstrate that the PG framework accurately reproduces theoretical predictions and experimental observations, while offering a unified, mathematically consistent treatment of the constraints inherent to phase-field fracture modeling.

Alexandre Ern, Brendan Keith, Dohyun Kim et al.

We introduce a family of proximal discontinuous Galerkin methods for variational inequalities, focusing on the obstacle problem as a didactic example. Each member of this family is born from applying a different well-known nonconforming finite element discretization to the Bregman proximal point method. We explicitly treat four examples: the symmetric interior penalty discontinuous Galerkin, the enriched Galerkin, the hybridizable interior penalty and the hybrid high-order methods. We formulate a unified analysis framework for this family of methods and prove the existence and uniqueness of solutions, energy dissipation, and error estimates for both the primal and dual variables. Remarkably, the proximal hybrid high-order method with piecewise constant cell unknowns and piecewise affine cell unknowns leads to the first higher-order convergence result for any proximal Galerkin method.

Peter Sentz, Stanley Nicholson, Yujin Cho et al.

The characterization of Hamiltonians and other components of open quantum dynamical systems plays a crucial role in quantum computing and other applications. Scientific machine learning techniques have been applied to this problem in a variety of ways, including by modeling with deep neural networks. However, the majority of mathematical models describing open quantum systems are linear, and the natural nonlinearities in learnable models have not been incorporated using physical principles. We present a data-driven model for open quantum systems that includes learnable, thermodynamically consistent terms. The trained model is interpretable, as it directly estimates the system Hamiltonian and linear components of coupling to the environment. We validate the model on synthetic two and three-level data, as well as experimental two-level data collected from a quantum device at Lawrence Livermore National Laboratory.

Brendan Keith, Ustim Khristenko, Barbara Wohlmuth

We develop a novel data-driven approach to modeling the atmospheric boundary layer. This approach leads to a nonlocal, anisotropic synthetic turbulence model which we refer to as the deep rapid distortion (DRD) model. Our approach relies on an operator regression problem which characterizes the best fitting candidate in a general family of nonlocal covariance kernels parameterized in part by a neural network. This family of covariance kernels is expressed in Fourier space and is obtained from approximate solutions to the Navier--Stokes equations at very high Reynolds numbers. Each member of the family incorporates important physical properties such as mass conservation and a realistic energy cascade. The DRD model can be calibrated with noisy data from field experiments. After calibration, the model can be used to generate synthetic turbulent velocity fields. To this end, we provide a new numerical method based on domain decomposition which delivers scalable, memory-efficient turbulence generation with the DRD model as well as others. We demonstrate the robustness of our approach with both filtered and noisy data coming from the 1968 Air Force Cambridge Research Laboratory Kansas experiments. Using this data, we witness exceptional accuracy with the DRD model, especially when compared to the International Electrotechnical Commission standard.

Brendan Keith, Akshay Khadse, Scott E. Field

We introduce a gravitational waveform inversion strategy that discovers mechanical models of binary black hole (BBH) systems. We show that only a single time series of (possibly noisy) waveform data is necessary to construct the equations of motion for a BBH system. Starting with a class of universal differential equations parameterized by feed-forward neural networks, our strategy involves the construction of a space of plausible mechanical models and a physics-informed constrained optimization within that space to minimize the waveform error. We apply our method to various BBH systems including extreme and comparable mass ratio systems in eccentric and non-eccentric orbits. We show the resulting differential equations apply to time durations longer than the training interval, and relativistic effects, such as perihelion precession, radiation reaction, and orbital plunge, are automatically accounted for. The methods outlined here provide a new, data-driven approach to studying the dynamics of binary black hole systems.

Leszek Demkowicz, Jay Gopalakrishnan, Brendan Keith

This article introduces the DPG-star (from now on, denoted DPG$^*$) finite element method. It is a method that is in some sense dual to the discontinuous Petrov-Galerkin (DPG) method. The DPG methodology can be viewed as a means to solve an overdetermined discretization of a boundary value problem. In the same vein, the DPG$^*$ methodology is a means to solve an underdetermined discretization. These two viewpoints are developed by embedding the same operator equation into two different saddle-point problems. The analyses of the two problems have many common elements. Comparison to other methods in the literature round out the newly garnered perspective. Notably, DPG$^*$ and DPG methods can be seen as generalizations of $\mathcal{L}\mathcal{L}^\ast$ and least-squares methods, respectively. A priori error analysis and a posteriori error control for the DPG$^*$ method are considered in detail. Reports of several numerical experiments are provided which demonstrate the essential features of the new method. A notable difference between the results from the DPG$^*$ and DPG analyses is that the convergence rates of the former are limited by the regularity of an extraneous Lagrange multiplier variable.

Brendan Keith, Philipp Knechtges, Nathan V. Roberts et al.

We explore a vexing benchmark problem for viscoelastic fluid flows with the discontinuous Petrov-Galerkin (DPG) finite element method of Demkowicz and Gopalakrishnan [1,2]. In our analysis, we develop an intrinsic a posteriori error indicator which we use for adaptive mesh generation. The DPG method is useful for the problem we consider because the method is inherently stable---requiring no stabilization of the linearized discretization in order to handle the advective terms in the model. Because stabilization is a pressing issue in these models, this happens to become a very useful property of the method which simplifies our analysis. This built-in stability at all length scales and the a posteriori error indicator additionally allows for the generation of parameter-specific meshes starting from a common coarse initial mesh. A DPG discretization always produces a symmetric positive definite stiffness matrix. This feature allows us to use the most efficient direct solvers for all of our computations. We use the Camellia finite element software package [3,4] for all of our analysis.

Federico Fuentes, Brendan Keith, Leszek Demkowicz et al.

This article presents a general approach akin to domain-decomposition methods to solve a single linear PDE, but where each subdomain of a partitioned domain is associated to a distinct variational formulation coming from a mutually well-posed family of broken variational formulations of the original PDE. It can be exploited to solve challenging problems in a variety of physical scenarios where stability or a particular mode of convergence is desired in a part of the domain. The linear elasticity equations are solved in this work, but the approach can be applied to other equations as well. The broken variational formulations, which are essentially extensions of more standard formulations, are characterized by the presence of mesh-dependent broken test spaces and interface trial variables at the boundaries of the elements of the mesh. This allows necessary information to be naturally transmitted between adjacent subdomains, resulting in coupled variational formulations which are then proved to be globally well-posed. They are solved numerically using the DPG methodology, which is especially crafted to produce stable discretizations of broken formulations. Finally, expected convergence rates are verified in two different and illustrative examples.