AttendSeg: A Tiny Attention Condenser Neural Network for Semantic Segmentation on the Edge
This work addresses efficient semantic segmentation for TinyML applications on edge devices, representing an incremental improvement in model compression.
The authors tackled the problem of on-device semantic segmentation by introducing AttendSeg, a compact neural network that achieves comparable accuracy to larger models while requiring over 27x fewer MACs, 72x fewer parameters, and 288x lower weight memory.
In this study, we introduce \textbf{AttendSeg}, a low-precision, highly compact deep neural network tailored for on-device semantic segmentation. AttendSeg possesses a self-attention network architecture comprising of light-weight attention condensers for improved spatial-channel selective attention at a very low complexity. The unique macro-architecture and micro-architecture design properties of AttendSeg strike a strong balance between representational power and efficiency, achieved via a machine-driven design exploration strategy tailored specifically for the task at hand. Experimental results demonstrated that the proposed AttendSeg can achieve segmentation accuracy comparable to much larger deep neural networks with greater complexity while possessing a significantly lower architecture and computational complexity (requiring as much as >27x fewer MACs, >72x fewer parameters, and >288x lower weight memory requirements), making it well-suited for TinyML applications on the edge.