Performance Analysis of DNN Inference/Training with Convolution and non-Convolution Operations
This work addresses a gap in performance analysis for deep learning accelerators, particularly for training and non-convolution operations, which is important for hardware designers and researchers, though it appears incremental as it extends existing frameworks rather than introducing a new paradigm.
The paper tackles the limitations of existing performance analysis frameworks for deep neural network accelerators, which focus only on convolution layers and inference, by proposing SimDIT, a framework that covers both convolution and non-convolution operations for inference and training, revealing that non-convolution operations constitute 59.5% of runtime for ResNet-50 training and achieving an 18X performance improvement for ResNet-50 inference through optimized resource allocation.
Today's performance analysis frameworks for deep learning accelerators suffer from two significant limitations. First, although modern convolutional neural network (CNNs) consist of many types of layers other than convolution, especially during training, these frameworks largely focus on convolution layers only. Second, these frameworks are generally targeted towards inference, and lack support for training operations. This work proposes a novel performance analysis framework, SimDIT, for general ASIC-based systolic hardware accelerator platforms. The modeling effort of SimDIT comprehensively covers convolution and non-convolution operations of both CNN inference and training on a highly parameterizable hardware substrate. SimDIT is integrated with a backend silicon implementation flow and provides detailed end-to-end performance statistics (i.e., data access cost, cycle counts, energy, and power) for executing CNN inference and training workloads. SimDIT-enabled performance analysis reveals that on a 64X64 processing array, non-convolution operations constitute 59.5% of total runtime for ResNet-50 training workload. In addition, by optimally distributing available off-chip DRAM bandwidth and on-chip SRAM resources, SimDIT achieves 18X performance improvement over a generic static resource allocation for ResNet-50 inference.