Scaling Up Dynamic Graph Representation Learning via Spiking Neural Networks
This work addresses the problem of high computation and memory overheads for researchers and practitioners working with large temporal graphs, offering a more scalable solution.
The paper tackles the scalability challenge in dynamic graph representation learning by proposing SpikeNet, a framework that uses spiking neural networks (SNNs) instead of recurrent neural networks (RNNs) to model temporal graphs, achieving better performance on temporal node classification with lower computational costs, such as handling a graph with 2.7M nodes and 13.9M edges efficiently.
Recent years have seen a surge in research on dynamic graph representation learning, which aims to model temporal graphs that are dynamic and evolving constantly over time. However, current work typically models graph dynamics with recurrent neural networks (RNNs), making them suffer seriously from computation and memory overheads on large temporal graphs. So far, scalability of dynamic graph representation learning on large temporal graphs remains one of the major challenges. In this paper, we present a scalable framework, namely SpikeNet, to efficiently capture the temporal and structural patterns of temporal graphs. We explore a new direction in that we can capture the evolving dynamics of temporal graphs with spiking neural networks (SNNs) instead of RNNs. As a low-power alternative to RNNs, SNNs explicitly model graph dynamics as spike trains of neuron populations and enable spike-based propagation in an efficient way. Experiments on three large real-world temporal graph datasets demonstrate that SpikeNet outperforms strong baselines on the temporal node classification task with lower computational costs. Particularly, SpikeNet generalizes to a large temporal graph (2.7M nodes and 13.9M edges) with significantly fewer parameters and computation overheads.Our code is publicly available at \url{https://github.com/EdisonLeeeee/SpikeNet}.