Not All Ops Are Created Equal!
This work addresses the need for more accurate efficiency metrics for deploying neural networks on mobile and embedded devices, though it is incremental as it builds on existing design practices.
The paper tackled the problem that typical neural network efficiency metrics like total operations and parameters do not accurately correlate with actual deployment metrics such as energy and memory footprint, showing that throughput and energy can vary by up to 5X across different operation types on an Arm Cortex-M7 microcontroller.
Efficient and compact neural network models are essential for enabling the deployment on mobile and embedded devices. In this work, we point out that typical design metrics for gauging the efficiency of neural network architectures -- total number of operations and parameters -- are not sufficient. These metrics may not accurately correlate with the actual deployment metrics such as energy and memory footprint. We show that throughput and energy varies by up to 5X across different neural network operation types on an off-the-shelf Arm Cortex-M7 microcontroller. Furthermore, we show that the memory required for activation data also need to be considered, apart from the model parameters, for network architecture exploration studies.