Complexity bounds on neural networks for the solution of structured linear systems of equations
This work provides theoretical guarantees for neural network efficiency in solving linear systems common in numerical methods, but it is incremental as it extends existing matrix inversion results.
The paper tackles the problem of approximating solutions to structured linear systems using ReLU neural networks, deriving explicit upper bounds on network complexity in terms of matrix size and condition number.
We derive upper bounds on the complexity of ReLU neural networks approximating the solution of a linear system given the matrix and the right-hand side. We focus on matrices which are symmetric positive definite and sparse, as they appear in the context of finite difference and finite element methods. For such matrices, we extend available results for the matrix inversion to the task of solving a linear system, where we leverage favorable properties of classical methods such as the modified Richardson and the conjugate gradient method. Our bounds on the number of layers and neurons are not only explicit with respect to the size of the matrices, but also with respect to their condition numbers.