Training a Binary Weight Object Detector by Knowledge Transfer for Autonomous Driving
This addresses the need for small, energy-efficient models for real-time object detection in autonomous driving systems, representing an incremental improvement in low-precision neural network training.
The paper tackled the problem of training binary weight neural networks for object detection in autonomous driving, which suffer from accuracy degradation due to extreme low-bit representation, by proposing a knowledge transfer method using a full-precision teacher network; the result was high detection accuracy while reducing model sizes from 257 MB to 8.8 MB for DarkNet-YOLO and from 193 MB to 7.9 MB for MobileNet-YOLO on the KITTI benchmark.
Autonomous driving has harsh requirements of small model size and energy efficiency, in order to enable the embedded system to achieve real-time on-board object detection. Recent deep convolutional neural network based object detectors have achieved state-of-the-art accuracy. However, such models are trained with numerous parameters and their high computational costs and large storage prohibit the deployment to memory and computation resource limited systems. Low-precision neural networks are popular techniques for reducing the computation requirements and memory footprint. Among them, binary weight neural network (BWN) is the extreme case which quantizes the float-point into just $1$ bit. BWNs are difficult to train and suffer from accuracy deprecation due to the extreme low-bit representation. To address this problem, we propose a knowledge transfer (KT) method to aid the training of BWN using a full-precision teacher network. We built DarkNet- and MobileNet-based binary weight YOLO-v2 detectors and conduct experiments on KITTI benchmark for car, pedestrian and cyclist detection. The experimental results show that the proposed method maintains high detection accuracy while reducing the model size of DarkNet-YOLO from 257 MB to 8.8 MB and MobileNet-YOLO from 193 MB to 7.9 MB.