AMStraMGRAM: Adaptive Multi-cutoff Strategy Modification for ANaGRAM
This work addresses optimization difficulties in PINNs for scientific computing, offering an incremental enhancement to existing methods.
The paper tackles the challenge of training physics-informed neural networks (PINNs) by analyzing ANaGRAM, a natural-gradient method with cutoff regularization, and proposes an adaptive multi-cutoff strategy that improves performance, achieving machine precision on some benchmark PDE experiments.
Recent works have shown that natural gradient methods can significantly outperform standard optimizers when training physics-informed neural networks (PINNs). In this paper, we analyze the training dynamics of PINNs optimized with ANaGRAM, a natural-gradient-inspired approach employing singular value decomposition with cutoff regularization. Building on this analysis, we propose a multi-cutoff adaptation strategy that further enhances ANaGRAM's performance. Experiments on benchmark PDEs validate the effectiveness of our method, which allows to reach machine precision on some experiments. To provide theoretical grounding, we develop a framework based on spectral theory that explains the necessity of regularization and extend previous shown connections with Green's functions theory.