MinUn: Accurate ML Inference on Microcontrollers
This work addresses the challenge of deploying ML models on resource-constrained microcontrollers, which is crucial for enabling AI in edge devices, though it appears incremental as it builds on existing TinyML frameworks.
The authors tackled the problem of generating efficient machine learning inference code for microcontrollers by developing MinUn, a TinyML framework that addresses memory frugality, parametric number representation, precision assignment, and fragmentation avoidance, resulting in code that outperforms prior frameworks on ARM microcontrollers like Arduino Uno and STM32H747.
Running machine learning inference on tiny devices, known as TinyML, is an emerging research area. This task requires generating inference code that uses memory frugally, a task that standard ML frameworks are ill-suited for. A deployment framework for TinyML must be a) parametric in the number representation to take advantage of the emerging representations like posits, b) carefully assign high-precision to a few tensors so that most tensors can be kept in low-precision while still maintaining model accuracy, and c) avoid memory fragmentation. We describe MinUn, the first TinyML framework that holistically addresses these issues to generate efficient code for ARM microcontrollers (e.g., Arduino Uno, Due and STM32H747) that outperforms the prior TinyML frameworks.