Hardware Acceleration of Sampling Algorithms in Sample and Aggregate Graph Neural Networks
This addresses a critical performance limitation in GNN training for large datasets, though it is an incremental improvement focused on hardware optimization.
The paper tackles the bottleneck of slow sampling in Graph Neural Networks by proposing a new neighbor sampler (CONCAT Sampler) and a hardware accelerator, achieving a 300-1000x speedup in sampling time while maintaining test accuracy.
Sampling is an important process in many GNN structures in order to train larger datasets with a smaller computational complexity. However, compared to other processes in GNN (such as aggregate, backward propagation), the sampling process still costs tremendous time, which limits the speed of training. To reduce the time of sampling, hardware acceleration is an ideal choice. However, state of the art GNN acceleration proposal did not specify how to accelerate the sampling process. What's more, directly accelerating traditional sampling algorithms will make the structure of the accelerator very complicated. In this work, we made two contributions: (1) Proposed a new neighbor sampler: CONCAT Sampler, which can be easily accelerated on hardware level while guaranteeing the test accuracy. (2) Designed a CONCAT-sampler-accelerator based on FPGA, with which the neighbor sampling process boosted to about 300-1000 times faster compared to the sampling process without it.