Graph-based Neural Acceleration for Nonnegative Matrix Factorization
This work addresses the computational bottleneck in nonnegative matrix factorization for applications in sparse linear algebra and related fields, representing an incremental improvement by combining existing graph neural network concepts with matrix factorization methods.
The paper tackles the problem of accelerating nonnegative matrix factorization by introducing a graph-based neural acceleration technique that interleaves bipartite self-attention layers with updates from the alternating direction method of multipliers, achieving substantial acceleration on synthetic and real-world datasets.
We describe a graph-based neural acceleration technique for nonnegative matrix factorization that builds upon a connection between matrices and bipartite graphs that is well-known in certain fields, e.g., sparse linear algebra, but has not yet been exploited to design graph neural networks for matrix computations. We first consider low-rank factorization more broadly and propose a graph representation of the problem suited for graph neural networks. Then, we focus on the task of nonnegative matrix factorization and propose a graph neural network that interleaves bipartite self-attention layers with updates based on the alternating direction method of multipliers. Our empirical evaluation on synthetic and two real-world datasets shows that we attain substantial acceleration, even though we only train in an unsupervised fashion on smaller synthetic instances.