Multi-GPU SNN Simulation with Static Load Balancing
This simulator addresses the problem of scaling SNN simulations for researchers and practitioners working with large-scale neural networks, offering an incremental improvement in performance and resource efficiency.
The paper introduces a multi-GPU SNN simulator capable of handling millions of neurons and billions of synapses across 8 GPUs. It achieves faster simulation, lower memory consumption, and linear scaling with the number of GPUs compared to two state-of-the-art simulators.
We present a SNN simulator which scales to millions of neurons, billions of synapses, and 8 GPUs. This is made possible by 1) a novel, cache-aware spike transmission algorithm 2) a model parallel multi-GPU distribution scheme and 3) a static, yet very effective load balancing strategy. The simulator further features an easy to use API and the ability to create custom models. We compare the proposed simulator against two state of the art ones on a series of benchmarks using three well-established models. We find that our simulator is faster, consumes less memory, and scales linearly with the number of GPUs.