Polynomial-Spline Neural Networks with Exact Integrals
This addresses consistency and integration issues in scientific machine learning, though it is incremental as it builds on existing polynomial and spline methods.
The paper tackles the limitations of neural networks in solving variational problems by introducing a novel architecture that combines polynomial mixture-of-experts with B1-spline basis functions, enabling exact integration and achieving expected convergence rates in regression tasks.
Using neural networks to solve variational problems, and other scientific machine learning tasks, has been limited by a lack of consistency and an inability to exactly integrate expressions involving neural network architectures. We address these limitations by formulating a novel neural network architecture that combines a polynomial mixture-of-experts model with free knot B1-spline basis functions. Effectively, our architecture performs piecewise polynomial approximation on each cell of a trainable partition of unity. Our architecture exhibits both $h$- and $p$- refinement for regression problems at the convergence rates expected from approximation theory, allowing for consistency in solving variational problems. Moreover, this architecture, its moments, and its partial derivatives can all be integrated exactly, obviating a reliance on sampling or quadrature and enabling error-free computation of variational forms. We demonstrate the success of our network on a range of regression and variational problems that illustrate the consistency and exact integrability of our network architecture.