Scaling Laws and Pathologies of Single-Layer PINNs: Network Width and PDE Nonlinearity
This addresses optimization pathologies in PINNs for scientific computing, but it is incremental as it builds on known spectral bias concepts.
The paper tackled the problem of scaling failures in Single-Layer Physics-Informed Neural Networks (PINNs) on nonlinear PDEs, finding that solution error does not decrease with network width due to optimization bottlenecks, with errors exacerbated by nonlinearity.
We establish empirical scaling laws for Single-Layer Physics-Informed Neural Networks on canonical nonlinear PDEs. We identify a dual optimization failure: (i) a baseline pathology, where the solution error fails to decrease with network width, even at fixed nonlinearity, falling short of theoretical approximation bounds, and (ii) a compounding pathology, where this failure is exacerbated by nonlinearity. We provide quantitative evidence that a simple separable power law is insufficient, and that the scaling behavior is governed by a more complex, non-separable relationship. This failure is consistent with the concept of spectral bias, where networks struggle to learn the high-frequency solution components that intensify with nonlinearity. We show that optimization, not approximation capacity, is the primary bottleneck, and propose a methodology to empirically measure these complex scaling effects.