GAN-Knowledge Distillation for one-stage Object Detection
This work addresses the need for efficient object detection in resource-constrained environments, though it is incremental as it adapts existing GAN and distillation concepts to a specific domain.
The paper tackles the problem of knowledge distillation for one-stage object detectors by proposing a GAN-based method that uses teacher and student feature maps as true and fake samples for adversarial training, resulting in improved student network performance.
Convolutional neural networks have a significant improvement in the accuracy of Object detection. As convolutional neural networks become deeper, the accuracy of detection is also obviously improved, and more floating-point calculations are needed. Many researchers use the knowledge distillation method to improve the accuracy of student networks by transferring knowledge from a deeper and larger teachers network to a small student network, in object detection. Most methods of knowledge distillation need to designed complex cost functions and they are aimed at the two-stage object detection algorithm. This paper proposes a clean and effective knowledge distillation method for the one-stage object detection. The feature maps generated by teacher network and student network are used as true samples and fake samples respectively, and generate adversarial training for both to improve the performance of the student network in one-stage object detection.