Lightweight Geometric Adaptation for Training Physics-Informed Neural Networks
For researchers using PINNs to solve PDEs, this work provides a computationally efficient optimizer that enhances performance without requiring second-order matrices.
Physics-Informed Neural Networks (PINNs) suffer from slow convergence and instability due to challenging loss landscapes. The proposed lightweight curvature-aware optimization framework improves convergence speed, training stability, and accuracy across diverse PDE benchmarks, including the high-dimensional heat equation and Gray-Scott system.
Physics-Informed Neural Networks (PINNs) often suffer from slow convergence, training instability, and reduced accuracy on challenging partial differential equations due to the anisotropic and rapidly varying geometry of their loss landscapes. We propose a lightweight curvature-aware optimization framework that augments existing first-order optimizers with an adaptive predictive correction based on secant information. Consecutive gradient differences are used as a cheap proxy for local geometric change, together with a step-normalized secant curvature indicator to control the correction strength. The framework is plug-and-play, computationally efficient, and broadly compatible with existing optimizers, without explicitly forming second-order matrices. Experiments on diverse PDE benchmarks show consistent improvements in convergence speed, training stability, and solution accuracy over standard optimizers and strong baselines, including on the high-dimensional heat equation, Gray--Scott system, Belousov--Zhabotinsky system, and 2D Kuramoto--Sivashinsky system.