Gradient Scaling Effects in Adaptive Spectral PINNs for Stiff Nonlinear ODEs
For researchers using PINNs on stiff dynamical systems, this paper shows that IC embedding choice is not neutral and must be tuned for stiffness, though the findings are incremental and domain-specific.
This work investigates how initial-condition (IC) gating functions in adaptive spectral PINNs affect training on stiff nonlinear ODEs. It finds that exponential gating performs better at moderate stiffness (k=20) while linear gating is preferable at higher stiffness (k=60), with statistical significance confirmed by Wilcoxon tests.
Physics-Informed Neural Networks (PINNs) often struggle to train reliably on stiff and oscillatory dynamical systems due to poor optimization conditioning. While prior work has emphasized representational remedies such as spectral parameterizations, the optimization implications of initial-condition (IC) embeddings in adaptive spectral PINNs have not been well characterized. In this work, we show that the choice of IC gating function induces explicit time-dependent gradient scaling, which interacts with spectral representations during training. Using a nonlinear stiff spring-pendulum ODE as a controlled benchmark, we compare exponential and linear IC gates in combination with fixed and adaptive Fourier spectral trunks. We observe stiffness-dependent changes in relative dominance for adaptive PINNs: at moderate stiffness ($k=20$), exponential gating often yields lower error but exhibits heterogeneous behavior across random seeds, whereas at higher stiffness ($k=60$), linear gating becomes preferable, with additional reversals observed at larger $k$. These trends hold for both relative $L^2$ error and maximum pointwise error and are confirmed by paired Wilcoxon signed-rank tests with Holm correction. Overall, our results demonstrate that IC embeddings are not a neutral design choice in PINNs: the induced gradient scaling materially shapes optimization conditioning in stiff regimes, with distinct sensitivity patterns in baseline and adaptive spectral models.