Hierarchical Block Sparse Neural Networks
This addresses the efficiency-accuracy trade-off in sparse DNNs for machine learning practitioners, representing an incremental improvement over existing sparse methods.
The paper tackles the problem of sparse deep neural networks (DNNs) having poor runtime efficiency on regular hardware and suboptimal accuracy due to structural constraints, proposing Hierarchical Block sparse Neural Networks (HBsNN) that achieve better runtime performance than unstructured sparse models and better accuracy than highly structured sparse models for a given sparsity.
Sparse deep neural networks(DNNs) are efficient in both memory and compute when compared to dense DNNs. But due to irregularity in computation of sparse DNNs, their efficiencies are much lower than that of dense DNNs on regular parallel hardware such as TPU. This inefficiency leads to poor/no performance benefits for sparse DNNs. Performance issue for sparse DNNs can be alleviated by bringing structure to the sparsity and leveraging it for improving runtime efficiency. But such structural constraints often lead to suboptimal accuracies. In this work, we jointly address both accuracy and performance of sparse DNNs using our proposed class of sparse neural networks called HBsNN (Hierarchical Block sparse Neural Networks). For a given sparsity, HBsNN models achieve better runtime performance than unstructured sparse models and better accuracy than highly structured sparse models.