Neural Tangent Kernel of Neural Networks with Loss Informed by Differential Operators
This work addresses the theoretical understanding of training dynamics for physics-informed neural networks, which is incremental as it builds on existing NTK theory.
The authors tackled the spectral bias phenomenon in neural networks by extending neural tangent kernel (NTK) theory to networks with physics-informed loss, finding that differential operators in the loss generally do not accelerate eigenvalue decay or enhance spectral bias, with experimental validation provided.
Spectral bias is a significant phenomenon in neural network training and can be explained by neural tangent kernel (NTK) theory. In this work, we develop the NTK theory for deep neural networks with physics-informed loss, providing insights into the convergence of NTK during initialization and training, and revealing its explicit structure. We find that, in most cases, the differential operators in the loss function do not induce a faster eigenvalue decay rate and stronger spectral bias. Some experimental results are also presented to verify the theory.