Group Scissor: Scaling Neuromorphic Computing Design to Large Neural Networks
This work addresses hardware implementation challenges for neuromorphic computing, enabling more efficient deployment of large neural networks, though it appears incremental as it builds on existing low-rank approximation and pruning techniques.
The paper tackles the problem of scaling neuromorphic computing systems to large neural networks by addressing limited crossbar size and routing congestion, achieving significant reductions in crossbar area (e.g., to 13.62% for LeNet) and routing area (e.g., to 8.1% for LeNet) without accuracy loss.
Synapse crossbar is an elementary structure in Neuromorphic Computing Systems (NCS). However, the limited size of crossbars and heavy routing congestion impedes the NCS implementations of big neural networks. In this paper, we propose a two-step framework (namely, group scissor) to scale NCS designs to big neural networks. The first step is rank clipping, which integrates low-rank approximation into the training to reduce total crossbar area. The second step is group connection deletion, which structurally prunes connections to reduce routing congestion between crossbars. Tested on convolutional neural networks of LeNet on MNIST database and ConvNet on CIFAR-10 database, our experiments show significant reduction of crossbar area and routing area in NCS designs. Without accuracy loss, rank clipping reduces total crossbar area to 13.62\% and 51.81\% in the NCS designs of LeNet and ConvNet, respectively. Following rank clipping, group connection deletion further reduces the routing area of LeNet and ConvNet to 8.1\% and 52.06\%, respectively.