From Two-Stream to One-Stream: Efficient RGB-T Tracking via Mutual Prompt Learning and Knowledge Distillation
This work addresses the problem of efficient multi-modal tracking for computer vision applications, offering a significant speed improvement with minimal accuracy loss, though it is incremental in nature.
The paper tackled efficient RGB-T object tracking by designing a two-stream teacher model with cross-modal mutual prompt learning and using it to guide a one-stream student model via knowledge distillation, achieving the highest precision rate for the teacher and a student model with comparable precision but over three times faster inference speed.
Due to the complementary nature of visible light and thermal infrared modalities, object tracking based on the fusion of visible light images and thermal images (referred to as RGB-T tracking) has received increasing attention from researchers in recent years. How to achieve more comprehensive fusion of information from the two modalities at a lower cost has been an issue that researchers have been exploring. Inspired by visual prompt learning, we designed a novel two-stream RGB-T tracking architecture based on cross-modal mutual prompt learning, and used this model as a teacher to guide a one-stream student model for rapid learning through knowledge distillation techniques. Extensive experiments have shown that, compared to similar RGB-T trackers, our designed teacher model achieved the highest precision rate, while the student model, with comparable precision rate to the teacher model, realized an inference speed more than three times faster than the teacher model.(Codes will be available if accepted.)