Learn Singularly Perturbed Solutions via Homotopy Dynamics
This addresses a specific bottleneck in scientific machine learning for researchers and practitioners dealing with singularly perturbed problems, though it appears incremental as it builds on existing neural network methods for PDEs.
The paper tackles the challenge of training neural networks for singularly perturbed PDEs, which cause near-singularities in the loss function, by introducing a homotopy dynamics method that significantly accelerates convergence and improves accuracy.
Solving partial differential equations (PDEs) using neural networks has become a central focus in scientific machine learning. Training neural networks for singularly perturbed problems is particularly challenging due to certain parameters in the PDEs that introduce near-singularities in the loss function. In this study, we overcome this challenge by introducing a novel method based on homotopy dynamics to effectively manipulate these parameters. From a theoretical perspective, we analyze the effects of these parameters on training difficulty in these singularly perturbed problems and establish the convergence of the proposed homotopy dynamics method. Experimentally, we demonstrate that our approach significantly accelerates convergence and improves the accuracy of these singularly perturbed problems. These findings present an efficient optimization strategy leveraging homotopy dynamics, offering a robust framework to extend the applicability of neural networks for solving singularly perturbed differential equations.