Ensemble learning for Physics Informed Neural Networks: a Gradient Boosting approach
This work addresses a specific bottleneck in PINNs for solving complex PDEs, offering an incremental improvement through ensemble learning techniques.
The paper tackles the challenge of simulating multi-scale and singular perturbation problems with physics-informed neural networks (PINNs) by introducing a gradient boosting training paradigm, which uses a sequence of neural networks to achieve superior performance compared to traditional PINNs and finite element methods in various benchmarks.
While the popularity of physics-informed neural networks (PINNs) is steadily rising, to this date, PINNs have not been successful in simulating multi-scale and singular perturbation problems. In this work, we present a new training paradigm referred to as "gradient boosting" (GB), which significantly enhances the performance of physics informed neural networks (PINNs). Rather than learning the solution of a given PDE using a single neural network directly, our algorithm employs a sequence of neural networks to achieve a superior outcome. This approach allows us to solve problems presenting great challenges for traditional PINNs. Our numerical experiments demonstrate the effectiveness of our algorithm through various benchmarks, including comparisons with finite element methods and PINNs. Furthermore, this work also unlocks the door to employing ensemble learning techniques in PINNs, providing opportunities for further improvement in solving PDEs.