Self-adaptive weights based on balanced residual decay rate for physics-informed neural networks and deep operator networks
This work addresses the problem of improving accuracy and efficiency in physics-informed deep learning for solving partial differential equations, representing an incremental advancement over existing adaptive weighting methods.
The paper tackles the challenge of training physics-informed neural networks for complex problems by addressing the discrepancy in residual convergence rates across training points, proposing a pointwise adaptive weighting method that balances these rates. The result is a method that achieves high prediction accuracy, fast convergence, low computational cost, and ease of hyperparameter tuning, as demonstrated through extensive numerical comparisons with state-of-the-art methods.
Physics-informed deep learning has emerged as a promising alternative for solving partial differential equations. However, for complex problems, training these networks can still be challenging, often resulting in unsatisfactory accuracy and efficiency. In this work, we demonstrate that the failure of plain physics-informed neural networks arises from the significant discrepancy in the convergence rate of residuals at different training points, where the slowest convergence rate dominates the overall solution convergence. Based on these observations, we propose a pointwise adaptive weighting method that balances the residual decay rate across different training points. The performance of our proposed adaptive weighting method is compared with current state-of-the-art adaptive weighting methods on benchmark problems for both physics-informed neural networks and physics-informed deep operator networks. Through extensive numerical results we demonstrate that our proposed approach of balanced residual decay rates offers several advantages, including bounded weights, high prediction accuracy, fast convergence rate, low training uncertainty, low computational cost, and ease of hyperparameter tuning.