Hands-on Guidance for Distilling Object Detectors
This work addresses computational complexity in deploying object detectors, but it is incremental as it builds on existing distillation methods by incorporating feature hierarchy.
The authors tackled the problem of feature hierarchy neglect in knowledge distillation for object detectors by proposing Hands-on Guidance Distillation, which distills all stage features and focuses on essence, resulting in better performance on accuracy and speed trade-offs as shown in evaluations on VOC and COCO datasets.
Knowledge distillation can lead to deploy-friendly networks against the plagued computational complexity problem, but previous methods neglect the feature hierarchy in detectors. Motivated by this, we propose a general framework for detection distillation. Our method, called Hands-on Guidance Distillation, distills the latent knowledge of all stage features for imposing more comprehensive supervision, and focuses on the essence simultaneously for promoting more intense knowledge absorption. Specifically, a series of novel mechanisms are designed elaborately, including correspondence establishment for consistency, hands-on imitation loss measure and re-weighted optimization from both micro and macro perspectives. We conduct extensive evaluations with different distillation configurations over VOC and COCO datasets, which show better performance on accuracy and speed trade-offs. Meanwhile, feasibility experiments on different structural networks further prove the robustness of our HGD.