CMSIS-NN: Efficient Neural Network Kernels for Arm Cortex-M CPUs
This work addresses the problem of enabling efficient neural network inference on resource-constrained IoT edge devices, representing an incremental optimization for a specific hardware platform.
The paper tackled the challenge of running neural networks efficiently on Arm Cortex-M CPUs for IoT edge devices, achieving a 4.6x improvement in runtime/throughput and a 4.9x improvement in energy efficiency.
Deep Neural Networks are becoming increasingly popular in always-on IoT edge devices performing data analytics right at the source, reducing latency as well as energy consumption for data communication. This paper presents CMSIS-NN, efficient kernels developed to maximize the performance and minimize the memory footprint of neural network (NN) applications on Arm Cortex-M processors targeted for intelligent IoT edge devices. Neural network inference based on CMSIS-NN kernels achieves 4.6X improvement in runtime/throughput and 4.9X improvement in energy efficiency.