Feature Mapping in Physics-Informed Neural Networks (PINNs)
This work addresses convergence and expressivity issues in PINNs, offering a practical technique for researchers in scientific machine learning, though it is incremental as it builds on existing feature mapping approaches.
The paper investigates training dynamics in Physics-Informed Neural Networks (PINNs) using feature mapping, revealing limitations of Fourier-based methods in some physics scenarios and proposing conditionally positive definite Radial Basis Functions as a better alternative, with empirical results showing efficacy in diverse forward and inverse problem sets.
In this paper, the training dynamics of PINNs with a feature mapping layer via the limiting Conjugate Kernel and Neural Tangent Kernel is investigated, shedding light on the convergence of PINNs; Although the commonly used Fourier-based feature mapping has achieved great success, we show its inadequacy in some physics scenarios. Via these two scopes, we propose conditionally positive definite Radial Basis Function as a better alternative. Lastly, we explore the feature mapping numerically in wide neural networks. Our empirical results reveal the efficacy of our method in diverse forward and inverse problem sets. Composing feature functions is found to be a practical way to address the expressivity and generalisability trade-off, viz., tuning the bandwidth of the kernels and the surjectivity of the feature mapping function. This simple technique can be implemented for coordinate inputs and benefits the broader PINNs research.