Towards a Foundation Model for Physics-Informed Neural Networks: Multi-PDE Learning with Active Sampling
This work addresses the problem of generalizability and computational cost in physics-based deep learning for researchers and practitioners, though it is incremental as it builds on existing PINN and active learning methods.
The paper tackles the limitation of traditional Physics-Informed Neural Networks (PINNs) being designed for single PDEs by proposing a foundation model that solves multiple PDEs within a unified architecture, demonstrating improved sample efficiency with active learning strategies like Monte Carlo Dropout-based uncertainty sampling, achieving significant performance gains with fewer training samples (e.g., 10-50% of the full dataset).
Physics-Informed Neural Networks (PINNs) have emerged as a powerful framework for solving partial differential equations (PDEs) by embedding physical laws into neural network training. However, traditional PINN models are typically designed for single PDEs, limiting their generalizability across different physical systems. In this work, we explore the potential of a foundation PINN model capable of solving multiple PDEs within a unified architecture. We investigate the efficacy of a single PINN framework trained on four distinct PDEs-the Simple Harmonic Oscillator (SHO), the 1D Heat Equation, the 1D Wave Equation, and the 2D Laplace Equation, demonstrating its ability to learn diverse physical dynamics. To enhance sample efficiency, we incorporate Active Learning (AL) using Monte Carlo (MC) Dropout-based uncertainty estimation, selecting the most informative training samples iteratively. We evaluate different active learning strategies, comparing models trained on 10%, 20%, 30%, 40%, and 50% of the full dataset, and analyze their impact on solution accuracy. Our results indicate that targeted uncertainty sampling significantly improves performance with fewer training samples, leading to efficient learning across multiple PDEs. This work highlights the feasibility of a generalizable PINN-based foundation model, capable of adapting to different physics-based problems without redesigning network architectures. Our findings suggest that multi-PDE PINNs with active learning can serve as an effective approach for reducing computational costs while maintaining high accuracy in physics-based deep learning applications.