YOLObile: Real-Time Object Detection on Mobile Devices via Compression-Compilation Co-Design
This work addresses the problem of real-time object detection for mobile applications, offering a significant speed improvement while maintaining accuracy, though it is incremental in nature.
The paper tackles the trade-off between accuracy and speed in object detection on mobile devices by proposing YOLObile, a framework that achieves 49.0 mAP with a 14x compression rate and 19.1 FPS inference speed, outperforming YOLOv4 by 5x.
The rapid development and wide utilization of object detection techniques have aroused attention on both accuracy and speed of object detectors. However, the current state-of-the-art object detection works are either accuracy-oriented using a large model but leading to high latency or speed-oriented using a lightweight model but sacrificing accuracy. In this work, we propose YOLObile framework, a real-time object detection on mobile devices via compression-compilation co-design. A novel block-punched pruning scheme is proposed for any kernel size. To improve computational efficiency on mobile devices, a GPU-CPU collaborative scheme is adopted along with advanced compiler-assisted optimizations. Experimental results indicate that our pruning scheme achieves 14$\times$ compression rate of YOLOv4 with 49.0 mAP. Under our YOLObile framework, we achieve 17 FPS inference speed using GPU on Samsung Galaxy S20. By incorporating our proposed GPU-CPU collaborative scheme, the inference speed is increased to 19.1 FPS, and outperforms the original YOLOv4 by 5$\times$ speedup. Source code is at: \url{https://github.com/nightsnack/YOLObile}.