R-TOSS: A Framework for Real-Time Object Detection using Semi-Structured Pruning
This work addresses efficiency challenges for real-time object detection in autonomous vehicles, representing an incremental improvement over existing pruning techniques.
The paper tackles the high memory and computational overheads of object detectors in autonomous vehicles by introducing R-TOSS, a semi-structured pruning framework, achieving up to 4.4x compression, 2.15x speedup in inference time, and 57.01% decrease in energy usage on YOLOv5.
Object detectors used in autonomous vehicles can have high memory and computational overheads. In this paper, we introduce a novel semi-structured pruning framework called R-TOSS that overcomes the shortcomings of state-of-the-art model pruning techniques. Experimental results on the JetsonTX2 show that R-TOSS has a compression rate of 4.4x on the YOLOv5 object detector with a 2.15x speedup in inference time and 57.01% decrease in energy usage. R-TOSS also enables 2.89x compression on RetinaNet with a 1.86x speedup in inference time and 56.31% decrease in energy usage. We also demonstrate significant improvements compared to various state-of-the-art pruning techniques.