Distilling Future Temporal Knowledge with Masked Feature Reconstruction for 3D Object Detection
This addresses the challenge of enabling online models to learn future knowledge in autonomous driving, representing an incremental improvement over existing knowledge distillation methods.
The paper tackles the problem of transferring future frame knowledge from offline to online models in camera-based temporal 3D object detection for autonomous driving, proposing FTKD which achieves up to 1.3 mAP and 1.3 NDS gains on the nuScenes dataset without increasing inference cost.
Camera-based temporal 3D object detection has shown impressive results in autonomous driving, with offline models improving accuracy by using future frames. Knowledge distillation (KD) can be an appealing framework for transferring rich information from offline models to online models. However, existing KD methods overlook future frames, as they mainly focus on spatial feature distillation under strict frame alignment or on temporal relational distillation, thereby making it challenging for online models to effectively learn future knowledge. To this end, we propose a sparse query-based approach, Future Temporal Knowledge Distillation (FTKD), which effectively transfers future frame knowledge from an offline teacher model to an online student model. Specifically, we present a future-aware feature reconstruction strategy to encourage the student model to capture future features without strict frame alignment. In addition, we further introduce future-guided logit distillation to leverage the teacher's stable foreground and background context. FTKD is applied to two high-performing 3D object detection baselines, achieving up to 1.3 mAP and 1.3 NDS gains on the nuScenes dataset, as well as the most accurate velocity estimation, without increasing inference cost.