Sparse matrix factorizations for fast linear solvers with application to Laplacian systems
For researchers in numerical linear algebra and graph algorithms, this work provides a novel framework that unifies and generalizes existing approaches to fast linear solvers.
The paper proposes a method to perform cheap iterations along non-sparse search directions by extracting them from a new sparse matrix factorization, achieving logarithmic cost per iteration. This yields a nearly-linear time algorithm for approximating the minimal norm solution of graph incidence matrix systems, connecting to Laplacian solvers.
In solving a linear system with iterative methods, one is usually confronted with the dilemma of having to choose between cheap, inefficient iterates over sparse search directions (e.g., coordinate descent), or expensive iterates in well-chosen search directions (e.g., conjugate gradients). In this paper, we propose to interpolate between these two extremes, and show how to perform cheap iterations along non-sparse search directions, provided that these directions can be extracted from a new kind of sparse factorization. For example, if the search directions are the columns of a hierarchical matrix, then the cost of each iteration is typically logarithmic in the number of variables. Using some graph-theoretical results on low-stretch spanning trees, we deduce as a special case a nearly-linear time algorithm to approximate the minimal norm solution of a linear system $Bx= b$ where $B$ is the incidence matrix of a graph. We thereby can connect our results to recently proposed nearly-linear time solvers for Laplacian systems, which emerge here as a particular application of our sparse matrix factorization.