Differentiable Network Pruning for Microcontrollers
It addresses the problem of on-device deep learning inference for embedded and IoT devices, offering an incremental improvement over existing MCU-specific compression techniques.
The paper tackles the challenge of deploying deep learning on resource-constrained microcontrollers by introducing a differentiable structured pruning method that integrates MCU-specific resource usage and parameter importance, resulting in models with up to 80x improved resource usage and 1.4x better resource usage in less time compared to prior methods.
Embedded and personal IoT devices are powered by microcontroller units (MCUs), whose extreme resource scarcity is a major obstacle for applications relying on on-device deep learning inference. Orders of magnitude less storage, memory and computational capacity, compared to what is typically required to execute neural networks, impose strict structural constraints on the network architecture and call for specialist model compression methodology. In this work, we present a differentiable structured network pruning method for convolutional neural networks, which integrates a model's MCU-specific resource usage and parameter importance feedback to obtain highly compressed yet accurate classification models. Our methodology (a) improves key resource usage of models up to 80x; (b) prunes iteratively while a model is trained, resulting in little to no overhead or even improved training time; (c) produces compressed models with matching or improved resource usage up to 1.4x in less time compared to prior MCU-specific methods. Compressed models are available for download.