CPUBone: Efficient Vision Backbone Design for Devices with Low Parallelization Capabilities
This work addresses the need for optimized vision models for CPU-based inference, which is an incremental improvement targeting embedded systems and devices with limited parallel processing.
The paper tackles the problem of designing efficient vision backbones for CPUs with low parallelization capabilities by modifying standard convolutions to reduce computational cost while preserving hardware-efficiency, resulting in CPUBone achieving state-of-the-art speed-accuracy trade-offs across diverse CPU devices and effective transfer to downstream tasks.
Recent research on vision backbone architectures has predominantly focused on optimizing efficiency for hardware platforms with high parallel processing capabilities. This category increasingly includes embedded systems such as mobile phones and embedded AI accelerator modules. In contrast, CPUs do not have the possibility to parallelize operations in the same manner, wherefore models benefit from a specific design philosophy that balances amount of operations (MACs) and hardware-efficient execution by having high MACs per second (MACpS). In pursuit of this, we investigate two modifications to standard convolutions, aimed at reducing computational cost: grouping convolutions and reducing kernel sizes. While both adaptations substantially decrease the total number of MACs required for inference, sustaining low latency necessitates preserving hardware-efficiency. Our experiments across diverse CPU devices confirm that these adaptations successfully retain high hardware-efficiency on CPUs. Based on these insights, we introduce CPUBone, a new family of vision backbone models optimized for CPU-based inference. CPUBone achieves state-of-the-art Speed-Accuracy Trade-offs (SATs) across a wide range of CPU devices and effectively transfers its efficiency to downstream tasks such as object detection and semantic segmentation. Models and code are available at https://github.com/altair199797/CPUBone.