Technical Report for ICCV 2021 Challenge SSLAD-Track3B: Transformers Are Better Continual Learners
This work addresses catastrophic forgetting in object detection for computer vision applications, presenting an incremental improvement over existing methods.
The paper tackles continual learning for object detection by proposing COLT, a transformer-based method with knowledge distillation and head expansion, achieving 70.78 mAP on the SSLAD-Track 3B challenge test set.
In the SSLAD-Track 3B challenge on continual learning, we propose the method of COntinual Learning with Transformer (COLT). We find that transformers suffer less from catastrophic forgetting compared to convolutional neural network. The major principle of our method is to equip the transformer based feature extractor with old knowledge distillation and head expanding strategies to compete catastrophic forgetting. In this report, we first introduce the overall framework of continual learning for object detection. Then, we analyse the key elements' effect on withstanding catastrophic forgetting in our solution. Our method achieves 70.78 mAP on the SSLAD-Track 3B challenge test set.