Ignacio Muga

Carlos Uriarte, David Pardo, Ignacio Muga et al.

Residual minimization is a widely used technique for solving Partial Differential Equations in variational form. It minimizes the dual norm of the residual, which naturally yields a saddle-point (min-max) problem over the so-called trial and test spaces. In the context of neural networks, we can address this min-max approach by employing one network to seek the trial minimum, while another network seeks the test maximizers. However, the resulting method is numerically unstable as we approach the trial solution. To overcome this, we reformulate the residual minimization as an equivalent minimization of a Ritz functional fed by optimal test functions computed from another Ritz functional minimization. We call the resulting scheme the Deep Double Ritz Method (D$^2$RM), which combines two neural networks for approximating trial functions and optimal test functions along a nested double Ritz minimization strategy. Numerical results on different diffusion and convection problems support the robustness of our method, up to the approximation properties of the networks and the training capacity of the optimizers.

Ignacio Muga, Kristoffer G. van der Zee

This work presents a comprehensive discretization theory for abstract linear operator equations in Banach spaces. The fundamental starting point of the theory is the idea of residual minimization in dual norms, and its inexact version using discrete dual norms. It is shown that this development, in the case of strictly-convex reflexive Banach spaces with strictly-convex dual, gives rise to a class of nonlinear Petrov-Galerkin methods and, equivalently, abstract mixed methods with monotone nonlinearity. Crucial in the formulation of these methods is the (nonlinear) bijective duality map. Under the Fortin condition, we prove discrete stability of the abstract inexact method, and subsequently carry out a complete error analysis. As part of our analysis, we prove new bounds for best-approximation projectors, which involve constants depending on the geometry of the underlying Banach space. The theory generalizes and extends the classical Petrov-Galerkin method as well as existing residual-minimization approaches, such as the discontinuous Petrov-Galerkin method.

Ignacio Muga, Matthew J. W. Tyler, Kristoffer G. van der Zee

We propose and analyse a minimal-residual method in discrete dual norms for approximating the solution of the advection-reaction equation in a weak Banach-space setting. The weak formulation allows for the direct approximation of solutions in the Lebesgue $L^p$-space, $1<p<\infty$. The greater generality of this weak setting is natural when dealing with rough data and highly irregular solutions, and when enhanced qualitative features of the approximations are needed. We first present a rigorous analysis of the well-posedness of the underlying continuous weak formulation, under natural assumptions on the advection-reaction coefficients. The main contribution is the study of several discrete subspace pairs guaranteeing the discrete stability of the method and quasi-optimality in $L^p$, and providing numerical illustrations of these findings, including the elimination of Gibbs phenomena, computation of optimal test spaces, and application to 2-D advection.

Tomasz Sluzalec, Mateusz Dobija, Anna Paszynska et al.

Petrov-Galerkin formulations with optimal test functions allow for the stabilization of finite element simulations. In particular, given a discrete trial space, the optimal test space induces a numerical scheme delivering the best approximation in terms of a problem-dependent energy norm. This ideal approach has two shortcomings: first, we need to explicitly know the set of optimal test functions; and second, the optimal test functions may have large supports inducing expensive dense linear systems. Nevertheless, parametric families of PDEs are an example where it is worth investing some (offline) computational effort to obtain stabilized linear systems that can be solved efficiently, for a given set of parameters, in an online stage. Therefore, as a remedy for the first shortcoming, we explicitly compute (offline) a function mapping any PDE-parameter, to the matrix of coefficients of optimal test functions (in a basis expansion) associated with that PDE-parameter. Next, as a remedy for the second shortcoming, we use the low-rank approximation to hierarchically compress the (non-square) matrix of coefficients of optimal test functions. In order to accelerate this process, we train a neural network to learn a critical bottleneck of the compression algorithm (for a given set of PDE-parameters). When solving online the resulting (compressed) Petrov-Galerkin formulation, we employ a GMRES iterative solver with inexpensive matrix-vector multiplications thanks to the low-rank features of the compressed matrix. We perform experiments showing that the full online procedure as fast as the original (unstable) Galerkin approach. In other words, we get the stabilization with hierarchical matrices and neural networks practically for free. We illustrate our findings by means of 2D Eriksson-Johnson and Hemholtz model problems.

Ignacio Muga, Jorge Perera, Sergio Rojas et al.

In this work, we propose and analyze a residual-minimization strategy for the numerical solution of nonlinear PDEs posed in Banach spaces. Given a finite-dimensional trial space and a suitably enriched discrete test space (of higher dimension than the trial space), we approximate the solution by minimizing the variational residual in a discrete dual norm. This minimization is equivalent to a nonlinear saddle-point formulation for the discrete solution in the trial space together with a residual representative in the test space. The latter provides a natural a posteriori error estimator, enabling automatic mesh adaptivity. To solve the resulting nonlinear saddle-point problem, we propose a Newton iteration whose linearized saddle-point system is symmetric, thereby guaranteeing solvability at each step. We take the $p$-Laplacian as a model problem and support the theoretical developments with representative numerical experiments, using standard $H^1$-conforming piecewise linear functions for the trial space, and lowest-order Crouzeix--Raviart functions for the test space.

Pablo Herrera, Jamie M. Taylor, Carlos Uriarte et al.

Solving Partial Differential Equations (PDEs) using neural networks presents different challenges, including integration errors and spectral bias, often leading to poor approximations. In addition, standard neural network-based methods, such as Physics-Informed Neural Networks (PINNs), often lack stability when dealing with PDEs characterized by low-regularity solutions. To address these limitations, we introduce the Ritz--Uzawa Neural Networks (RUNNs) framework, an iterative methodology to solve strong, weak, and ultra-weak variational formulations. Rewriting the PDE as a sequence of Ritz-type minimization problems within a Uzawa loop provides an iterative framework that, in specific cases, reduces both bias and variance during training. We demonstrate that the strong formulation offers a passive variance reduction mechanism, whereas variance remains persistent in weak and ultra-weak regimes. Furthermore, we address the spectral bias of standard architectures through a data-driven frequency tuning strategy. By initializing a Sinusoidal Fourier Feature Mapping based on the Normalized Cumulative Power Spectral Density (NCPSD) of previous residuals or their proxies, the network dynamically adapts its bandwidth to capture high-frequency components and severe singularities. Numerical experiments demonstrate the robustness of RUNNs, accurately resolving highly oscillatory solutions and successfully recovering a discontinuous $L^2$ solution from a distributional $H^{-2}$ source -- a scenario where standard energy-based methods fail.