Effectiveness of Function Matching in Driving Scene Recognition
This work addresses the need for efficient scene recognition in autonomous driving, but it is incremental as it applies an existing concept to a new domain.
The study tackled the problem of improving compact recognizers for autonomous driving by applying function matching knowledge distillation with massive unlabeled data, resulting in dramatic performance gains where the student model matched the teacher's performance.
Knowledge distillation is an effective approach for training compact recognizers required in autonomous driving. Recent studies on image classification have shown that matching student and teacher on a wide range of data points is critical for improving performance in distillation. This concept (called function matching) is suitable for driving scene recognition, where generally an almost infinite amount of unlabeled data are available. In this study, we experimentally investigate the impact of using such a large amount of unlabeled data for distillation on the performance of student models in structured prediction tasks for autonomous driving. Through extensive experiments, we demonstrate that the performance of the compact student model can be improved dramatically and even match the performance of the large-scale teacher by knowledge distillation with massive unlabeled data.