RUNNs: Ritz-Uzawa Neural Networks for Solving Variational Problems
This addresses stability and accuracy issues in neural network-based PDE solvers for researchers in computational science, though it appears incremental as it builds on existing variational methods.
The paper tackles challenges in solving PDEs with neural networks, such as integration errors and spectral bias, by introducing Ritz-Uzawa Neural Networks (RUNNs), an iterative framework that reduces bias and variance and accurately resolves oscillatory solutions, including recovering a discontinuous L^2 solution from an H^{-2} source where standard methods fail.
Solving Partial Differential Equations (PDEs) using neural networks presents different challenges, including integration errors and spectral bias, often leading to poor approximations. In addition, standard neural network-based methods, such as Physics-Informed Neural Networks (PINNs), often lack stability when dealing with PDEs characterized by low-regularity solutions. To address these limitations, we introduce the Ritz--Uzawa Neural Networks (RUNNs) framework, an iterative methodology to solve strong, weak, and ultra-weak variational formulations. Rewriting the PDE as a sequence of Ritz-type minimization problems within a Uzawa loop provides an iterative framework that, in specific cases, reduces both bias and variance during training. We demonstrate that the strong formulation offers a passive variance reduction mechanism, whereas variance remains persistent in weak and ultra-weak regimes. Furthermore, we address the spectral bias of standard architectures through a data-driven frequency tuning strategy. By initializing a Sinusoidal Fourier Feature Mapping based on the Normalized Cumulative Power Spectral Density (NCPSD) of previous residuals or their proxies, the network dynamically adapts its bandwidth to capture high-frequency components and severe singularities. Numerical experiments demonstrate the robustness of RUNNs, accurately resolving highly oscillatory solutions and successfully recovering a discontinuous $L^2$ solution from a distributional $H^{-2}$ source -- a scenario where standard energy-based methods fail.