Semialgebraic Neural Networks: From roots to representations
This work addresses a domain-specific problem in scientific computing for representing semialgebraic functions, but it is incremental as it builds on existing neural network and homotopy continuation methods.
The authors tackled the problem of representing bounded semialgebraic functions, which are common in scientific computing, by introducing Semialgebraic Neural Networks (SANNs) that can compute these functions up to the accuracy of a numerical ODE solver.
Many numerical algorithms in scientific computing -- particularly in areas like numerical linear algebra, PDE simulation, and inverse problems -- produce outputs that can be represented by semialgebraic functions; that is, the graph of the computed function can be described by finitely many polynomial equalities and inequalities. In this work, we introduce Semialgebraic Neural Networks (SANNs), a neural network architecture capable of representing any bounded semialgebraic function, and computing such functions up to the accuracy of a numerical ODE solver chosen by the programmer. Conceptually, we encode the graph of the learned function as the kernel of a piecewise polynomial selected from a class of functions whose roots can be evaluated using a particular homotopy continuation method. We show by construction that the SANN architecture is able to execute this continuation method, thus evaluating the learned semialgebraic function. Furthermore, the architecture can exactly represent even discontinuous semialgebraic functions by executing a continuation method on each connected component of the target function. Lastly, we provide example applications of these networks and show they can be trained with traditional deep-learning techniques.