FusedMM: A Unified SDDMM-SpMM Kernel for Graph Embedding and Graph Neural Networks
This work addresses performance bottlenecks for researchers and practitioners using graph algorithms, though it is incremental as it optimizes existing computational patterns.
The paper tackles the computational inefficiency in graph embedding and GNNs by developing FusedMM, a unified kernel that combines SDDMM and SpMM operations, resulting in an order of magnitude faster performance than existing kernels and up to 28x speedup in end-to-end graph embedding.
We develop a fused matrix multiplication kernel that unifies sampled dense-dense matrix multiplication and sparse-dense matrix multiplication under a single operation called FusedMM. By using user-defined functions, FusedMM can capture almost all computational patterns needed by popular graph embedding and GNN approaches. FusedMM is an order of magnitude faster than its equivalent kernels in Deep Graph Library. The superior performance of FusedMM comes from the low-level vectorized kernels, a suitable load balancing scheme and an efficient utilization of the memory bandwidth. FusedMM can tune its performance using a code generator and perform equally well on Intel, AMD and ARM processors. FusedMM speeds up an end-to-end graph embedding algorithm by up to 28x on different processors.