Architectural Strategies for the optimization of Physics-Informed Neural Networks
This work addresses optimization difficulties in PINNs for solving partial differential equations, which is an incremental improvement for researchers in scientific machine learning.
The paper tackles the notorious training challenges of Physics-Informed Neural Networks (PINNs) by analyzing optimization from a neural architecture perspective, revealing that Gaussian activations outperform alternatives and introducing a preconditioned architecture that enhances training, with validation on established PDEs.
Physics-informed neural networks (PINNs) offer a promising avenue for tackling both forward and inverse problems in partial differential equations (PDEs) by incorporating deep learning with fundamental physics principles. Despite their remarkable empirical success, PINNs have garnered a reputation for their notorious training challenges across a spectrum of PDEs. In this work, we delve into the intricacies of PINN optimization from a neural architecture perspective. Leveraging the Neural Tangent Kernel (NTK), our study reveals that Gaussian activations surpass several alternate activations when it comes to effectively training PINNs. Building on insights from numerical linear algebra, we introduce a preconditioned neural architecture, showcasing how such tailored architectures enhance the optimization process. Our theoretical findings are substantiated through rigorous validation against established PDEs within the scientific literature.