TransNet: Transferable Neural Networks for Partial Differential Equations
This work addresses the challenge of developing pre-trained neural networks for solving a wide class of PDEs, which is incremental as it builds on existing transfer learning approaches but reduces the need for target PDE information.
The paper tackles the problem of transfer learning for partial differential equations (PDEs) by proposing a method to construct transferable neural feature spaces without using PDE information, resulting in significantly improved transferability and several orders of magnitude smaller mean squared error than state-of-the-art methods.
Transfer learning for partial differential equations (PDEs) is to develop a pre-trained neural network that can be used to solve a wide class of PDEs. Existing transfer learning approaches require much information of the target PDEs such as its formulation and/or data of its solution for pre-training. In this work, we propose to construct transferable neural feature spaces from purely function approximation perspectives without using PDE information. The construction of the feature space involves re-parameterization of the hidden neurons and uses auxiliary functions to tune the resulting feature space. Theoretical analysis shows the high quality of the produced feature space, i.e., uniformly distributed neurons. Extensive numerical experiments verify the outstanding performance of our method, including significantly improved transferability, e.g., using the same feature space for various PDEs with different domains and boundary conditions, and the superior accuracy, e.g., several orders of magnitude smaller mean squared error than the state of the art methods.